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ABSTRACT 

This study examined the construct validity of 
cognitive structures, detined as sets of relationships among 
concepts. The investigaticn proceeded by multidimensional 
scaling analysis of concept similarity data obtained from 78 
grade nine science and 54 grade 12 biology students. Using a 
set of 15 syntactical concepts from the domain of the 
scientific method {e.g., "conclusion" or “hypothesis"), 
subjects responded to three concept rating tests: a word 
association test (constrained to "scientific meanings"), a 
Similarity judgment test, and a semantic differential test. 
Data were also gathered on the subjects in the form of tests 
of developmental level, field independence-dependence 
(Hidden Figures Test), and cognitive complexity. 

Validity was investigated through the scaling 
technigue of Kruskal applied to group average data, and 
through subgroup formation based upon individual differences 
in Similarity judgments as revealed by Tucker and Messick 
points-of-view analysis and Carroll and Chang individual 
differences analysis. The measure of convergence of scaling 
solutions waS a non-metric version of the orthogonal 
Procrustes rotation. 

It was hypothesized that individual differences in 
cognitive structures would be linked to individual 


differences in personality variables. In addition to the 
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three tests of personality variables administered, other 
variables investigated inciuded age, sex, grade, 
achievement, etiete and several variables relating to 
personal consistency of response on the similarity measures 
and fluency of response on the word association test. 

Ali configurations reported are in three dimensions. 
The goodness-of-fit, taken as evidence of convergent 
validity, was quite high between the group average data of 
both grades for the word association test and the similarity 
judgment test {average cr=0.90). Goodness-of-fit was 
considerably lower (average r=0.67) £or the semantic 
differential data. Both the Tucker and Messick technique and 
the Carroll and Chang technique produced subgroups with 
interpretable configuraticns, indicating evidence for the 
construct validity of cognitive structure. Measures of 
goodness-of-fit between these subgroup solutions were 
Peo (average r=0.67), indicating some evidence for 
individual differences in cognitive structures. Subgroups 
formed by the two scaling techniques bore no relationship to 
each other, but subgroups formed cn different similarity 
measures using the Carroll and Chang method bore strong 
resemblances to each other. Interpretability of Jenene 
configurations produced by the Kruskal technique was low, 
possibly due to high error in the data. The Carroll and 
Chang technigue, however, produced interpretable resuits, 
and thus was judged relatively robust with respect to error. 


The study provided some construct validity for the 
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anterpretation of configurations as cognitive structures 
based upon data obtained by word association and similarity 
judgment. Individual differences in cognitive structure were 
found in the configurations of the subgroups formed. 

None of the personality variables examined was 
fruitful in identifying members of subgroups formed. With 
the exception of the ccgnitive complexity test, 
reliabilities of the personality variables were 
satisfactory, with estimates by various methods ranging from 
0.50 to 0.76. There was some consistency of subgroup 
membership within the subgreups formed by the Carroll and 
Chang analysis, but not among those formed by the Tucker and 
Messick analysis on the different data sets. It is suggested 
that further progress in the field depends upcn refinement 
of the input to the scaling techniques, rather than 
refinement of the techniques themselves. Conceptual and 
empirical clarification of relevant personality variables is 


also a priority. 
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CHAPTER 1 


INTRODUCTION 


1.1 Introduction 

Chapter 1 presents an outline of the study. It 
includes a discussion of the purpose and significance of the 
study, a statement of the specific problem to be addressed, 
clarifications and definitions of terms, a brief summary of 
the procedure employed, and a statement of testable 


hypotheses. 


1.2 Purpose and Overview 

The purpose of this investigation was to assess the 
validity of three methods of uncovering cognitive structure 
which may describe certain interrelations among a_ set of 
science concepts. Campbell and Fiske (1959) maintain that 
evidence for construct validity of a method or instrument is 
provided when different instruments presumed to 
independently measure the same construct converge to the 
same result (convergent validity), and instruments 
hypothesized to be measuring different constructs 
discriminate between these constructs by yielding different 
results (discriminant validity). The nature of the term 


independent will be discussed below. Convergent validity of 
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the instrument used to determine cognitive structures was 
assessed through the sets of relationships perceived by the 
same groups of people on different similarity rating tasks 
analysed by the same method, and the sets of relationships 
produced by the same groups of people on the same similarity 
task analysed by different methods. Discriminant validity of 
the cognitive structures was indirectly assessed through the 
sets of relationships perceived by different groups of 
people using the same similarity rating tasks analysed by 
the same method. All comparisons of cognitive structures 
were done through goodness-of-fit measures between 
multidimensional configurations of concepts hypothesized to 
represent the cognitive structures. Interpretations of 
differences in the sets of relationships were attempted by 
relating differences among concept perceptions to 
differences among groups on the personality variables 
studied. The construct validation may be viewed as a two 
step process. The first step validates the methods of 
gathering similarity ratings. This may be attacked through 
analysis of full group average data across the different 
Similarity rating instruments. The second step is a 
validation of the structure itself, in the sense of showing 
that, in some sense, the recovered scaling solutions 
represent some psychological reality for the subject. This 
problem is attacked through analysis of the personality 
variables and the structures uncovered for subgroups of the 


sample. These personality variables include developmental 
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level, field independence, and cognitive complexity. 
Cognitive structure is viewed as an organization 
within a subject's memory of a set of concepts. The 
organization is governed by the perceived relationships 
among the concepts. The existence of cognitive structure is 
inferred from evidence gathered from the subject concerning 
his perceptions of the concepts and the relationships among 
them. This study focuses upon a set of 15 syntactical {as 
opposed to substantive) concepts within the domain of the 
scientific method (for example, "hypothesis" or "conclusion" 
as opposed to "energy" or "atom"). Data on perceived concept 
relationships were obtained from subjects by means of the 
methods of constrained association ({Garskof and Houston, 
1963), similarity judgment (Torgerson, 1958), and semantic 
differential (Osgood et al, 1957). Information obtained from 
subjects was processed by multidimensional scaling 
technigues (to be described in detail below) to yield 
configurations of the concepts represented as points in 
geometrical space, with distances between points 
corresponding to perceived dissimilarities among concepts. 
Besides the validity of cognitive structures, this study 
also addresses (although indirectly) the validity of 
representing these structures geometrically. Inferences are 
drawn from the multidimensional configurations to the 
organization of the concepts within the subjects' memories. 
Similarity of configurations obtained from analyses of data 


from the same subjects by different experimental techniques 
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is taken as evidence that the tests are tapping the same 
internal cognitive structure. Because of the nature of the 
data, configurations based upon group averages rather than 
individual data must be calculated. Averaging is necessary 
to allow random errors in individual data to cancel, 
producing data with less error and facilitating 
interpretation of the derived configurations. Since 
generalization to the perceptions of individuals is desired, 
the necessity of using grouped data imposes a limitation to 
the techniques used. Only by averaging and allowing some 
random error to cancel can the techniques produce 
interpretable results. 

The term constrained is used in several senses by 
different authors whose work is reported here. With 
reference to word association tests, the term constrained 
has heen used to mean any of the following: "respond with 
words you would use to define the stimulus word"; "produce 
exactly five responses"; or "respond with physics words 
only". In the present study, the term constrained was taken 
to mean "respond while thinking of the scientific meaning of 
the stimulus word". 

The constrained association task yielded a set of 
relatedness indices that were used as similarity measures 
for further analysis. The similarity judgment technique 
yielded a measure of similarity directly with no additional 
data manipulation. The semantic differential results were 


analysed by first producing an intercorrelation matrix for 
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the ratings of the concepts based upon the patterns of 
responses produced, and then using these intercorrelations 
as measures of similarity. 

Data gathered by the above methods were examined as 
follows. First, grouping the subjects by grade only (grades 
9 and 12 were used), the group average data for each grade 
were subjected to multidimensional scaling analysis 
{hereafter known as Kruskal scaling) (Kruskal, 1964a, 1964b) 
to estimate the overall dimensionality of the cognitive 
Structure of the entire grade group. Second, based upon the 
estimate of dimensionality obtained from the Kruskal 
analysis, the similarity matrix data were subjected to both 
points of view analysis (Tucker and Messick, 1963; Jackson 
and Messick, 1963) and individual differences analysis 
(Carroll and Chang, 1970; Carroll, 1972). These methods will 
be referred to as Tucker scaling and Carroll scaling, 
respectively. Kruskal analysis itself is not one of the 
methods under investigation. Preliminary investigation of 
dimensionality by this method was done for purposes of 
comparison of the group average results on the several data 
sets, and as a computer cost reducing device, since 
assessment of dimensionality by the Tucker or Carroll 
techniques is very costly. 

Kruskal scaling analysis has as its goal the 
uncovering of underlying dimensions that can be used to 
express geometrically the relationships among the data 


points under investigation. The Kruskal method is based upon 
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group or average data. The Tucker and Carroll analyses have 
Similar goals, with added facility to manipulate individual 
rather than group data. This facility allows the uncovering 
of differences among subjects in their perceptions of the 
relationships among the data points under investigation. The 
Tucker method uses an obverse principal components analysis 
of the similarity ratings by subjects matrix in order to 
characterize subject types according to their loadings on 
the factors produced. Kruskal analyses of the data produced 
by each of these types can then be compared. The Carroll 
analysis operates on a three dimensional array consisting of 
the similarity rating matrices obtained for each subject to 
produce a mapping of concepts into a geometrical space of 
any chosen number of dimensions, along with a subjects by 
dimensions set of weights that estimate the salience or 
importance of each of these dimensions to each subject in 
his individual estimates cf concept similarity. 

The Tucker method and the Carroll method have as 
+heir goal the revealing of individual differences in 
matrices of Similarity data. If the data meet the 
assumptions underlying both techniques the results should be 
comparable. Subgroups of subjects can be formed on the basis 
of the results of either analysis. Commonality of subgroup 
membership based upon each analysis may be taken as evidence 
of the applicability of both techniques to the data. 
However, lack of such commonality cannot, within the limits 


of the analyses undertaken herein, be used to show the 
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inapplicability of either method to the data. 


1.3 Statement of the Problem 
This study was an investigation of perceived 

Similarities among a set of 15 syntactical concepts from the 
domain of the scientific method. The same subjects were 
asked to respond in three different judgmental tasks to the 
same set of concepts. The questions addressed by this study 
are: 

(1) Are there commonalities among the configurations 

uncovered for entire groups of subjects by the three 

techniques? 

(2) Are there differences among subgroups in the 

configurations uncovered by either of the techniques 

which apply to individual data? 

(3) Can these differences be related in a meaningful 

Manner to personality variables measured independently 


of the similarity rating tasks? 


1.4 Significance of the Study 

The present study ascertains the degree of 
relationship among students! cognitive structures in the 
domain of syntactical science concepts as indicated by three 
judgmental tasks. According to Campbell and Fiske (1959) 
construct validity of instruments must be demonstrated by 
both convergent and discriminant validation processes. 


Pirst, construct validity of a hypothetical construct in a 
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given domain {cognitive structure) will be enhanced if the 
construct can be measured by experimentally independent 
means and if these independent measures of the construct 
correlate positively and significantly with each other. 
Second, construct validity is further enhanced if the 
measures of the construct correlate relatively poorly with 
measures of other constructs. The argument for convergent 
validity is straightforward, but that for discriminant 
validity is open to question. 

With respect to convergent validity, high 
correlation between tests which purport to measure the same 
construct may be taken as evidence of construct validity 
only if it can be established that the tests are independent 
of each other. That is, it must be established that the 
correlation between the tests is not an artifact of, for 
example, a common testing format, a common testing time, or 
some other extraneous variable. The meaning of independent, 
as used by Campbell and Fiske, must be made explicit. 
Clearly, two tests measuring in the same domain would have 
enough in common that too stringent an interpretation of the 
term independent would make convergent validity impossible. 
It is difficult to imagine tests aiming at the same 
construct but being completely unique in goals, methods, and 
assumptions. In the present context, the three tests used to 
obtain similarity information ma y be argued to be 
independent in a manner appropriate to the Campbell and 


Fiske usage. All three deal with syntactical concepts in the 
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domain of science, and are thus related. However, only in 
one of the tests, the similarity judgment test, would a 
naive subject even be aware that similarity information was 
being elicited. For the word association and semantic 
differential tests, the subject is asked to consider only 
one of the 15 concepts at a time. Further, in two of the 
tests, the similarity judgment and semantic differential, 
judgments are required, but in the case of the semantic 
differential, the subject is asked to relate one concept 
under study to adjectival pairs describing it, while in the 
Similarity judgment test, he is asked to relate two of the 
concepts under study. In the word association test, the 
subject is asked to associate to a concept word. It is hoped 
that the concepts presented will set the context enough that 
the subject would be operating in the scientific context 
rather than, for example, the legal (cf., "law" or 
"evidence") context. Word association tests come from a 
psychoanalytic tradition, similarity judgment from a 
psychophysical tradition, and semantic differential from a 
linguistic tradition. The three tests are similar enough to 
be within the same domain but different enough to qualify as 
independent for purposes of convergent validity. 

Campbell and Fiske's argument for discriminant 
validity has been criticised by Cronbach (1971). Cronbach 
maintained that it is not necessary for a pair of constructs 
to have low correlations between them in order that each be 


worth support as a scientific explanatory device. He argued 
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that any correlation substantially below 1.00 is sufficient 
to allow the possibility that both of two related constructs 
may be scientifically useful. This discussion of 
discriminant validity is not central to the present study in 
that, as will be discussed in detail below, it is not clear 
what would constitute establishment of discriminant validity 
of cognitive structure. 

‘The psychological significance of the study lies in 
its potential for establishing the construct validity of a 
view of cognitive structure as evidenced by a_ set of 
interrelations among concepts mapped onto a geometric space. 
Such validity can only help to bring conceptual and 
definitional clarity to the idea of cognitive structure, 
which appears in the literature in confusing and competing 
contexts (Ausubel, 1965; Bruner, 1960; Schwab, 1974.). 

The educational significance of the study pertains 


to the example concepts chosen for study. The concepts are: 


Conclusion Fact Law 
Discovery Hypothesis Preot 
Evidence Imagination Puzzle 
Experiment Interpretation Question 
Explanation Investigation Theory 


The particular concept domain investigated, that of 
syntactical concepts of the scientific method, has’ been 
chosen because of: (a) its applicability across the grade 


levels in question (grades 9 and 12), thus allowing 
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assessment of some developmental aspects of cognitive 
structure; the ease with which dimensions of the concept 
domain may be interpreted relative to other concept domains 
that have been investigated by similar methods (Kass, 19773 
Shavelson, 1972); and (c), the interest that the structure 
of scientific method itself has generated (Kuhn, 1962; 
Braithwaite, 1973). 

The cognitive structure of a student regarding 
scientific method has potential as both an independent and 
dependent variable in curriculum evaluation and design. It 
May be possible to judge the relationship of specific 
student characteristics, such as I.0., developmental level, 
or cognitive style to the cognitive structure among the 
above elements uncovered for that student. Although the 
educational import of this study lies principally in the 
application of the methodology to other, perhaps 
educationally more central concepts, further research with 
these concepts may reveal curricular consequences for the 
teaching of science. It may be possible to enhance the 
teaching of science by matching instruction to students! 
perceptions of the interrelationships among syntactical 
elements of science. Such a possibility is in line with 
Hunt's (1971) recommendation to match instruction to student 
characteristics. It may further be possible to design 
instruction to change a student's perception of the 
relationships among aspects of the scientific process. 


Research investigating the relationships among cognitive 
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structure, instruction, and achievement is reported in 
detail below (Johnson, 1964, 1965, 1967; Shavelson, 1972, 
1973501974}. 

The study has psychometric consequences in that it 
investigates a methodology for the uncovering of cognitive 
structure that has potential applicability to any set of 
related concepts, syntactical or substantive, within any 
domain of interest. 

This study is limited in that, although there is no 
logical reason for supposing a lack of connection between 
cognitive structure and science achievement, the cognitive 
Structure among the particular concepts studied cannot be 
predicted with any confidence to be a significant 
intervening variable for achievement in any common high 
school science course. Evidence has shown (Aikenhead, 1972; 
Simpson et al, 19723; MacDonald, 1978) little relationship 
between understanding the processes of science, however 
defined, and science achievement. Thus, unlike those studies 
which dealt with the structure within substantive concepts 
(Kass, 1971; Shavelson, 1972), no direct link with subject 
matter achievement can be expected from this study of 
syntactical concepts. However, this limitation is balanced 
by the advantage that these concepts, since they are not 
directly bound up in immediate classroom learning, may 
exhibit more individual variation in perception than would 
be expected from a set of concepts taken directly from 


recent or ongoing instruction. 
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1.5 Clarifications and Definitions 


Carroll (1964) defines concepts as "properties 
of...abstracted and often cognitively structured classes of 
‘mental’ experience." Experience in turn is defined as "any 
internal or perceptual response to stimulation". The term 
abstracted is the key to Carroll's definition of concept. 
Concepts are abstracted from a set of experienced exemplars. 
A necessary step in the formation of a concept is a 
differentiation of a subset of experience {exemplars of the 
concept) from the entire set of experience {both exemplars 
and non-exemplars of the concept). This generalization 
produces a very limited concept. A concept is further 
developed through its relations with other concepts. For 
example, the concept "stool" may be closely identified with 
the concept "chair", with the exception of the relational 
property "without a back". The concept "chair" in turn may 
be related to the more subsuming concept "furniture" by the 
relational property "for sitting on" . The set of 
relationships among concepts is defined as cognitive 
structure. 

For purposes of this study, a concept was delimited 
in terms of an educational concept rather than a laboratory 
concept. For Carroll, such a distinction is between a 


student learning the concept longitude and the concept of 
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learning to associate the nonsense syllable "DAX" as the 
name of a geometric shape of a certain colour. The concern 
of this study is entirely with educational concepts. 

1.52 The Nature of Structure 

This section will address the problem of defining 
the status of cognitive structure. The discussion will not 
focus on whether or not structure is best defined as 
hypothesis, theory, or model. Such a discussion would not 
prove worthwhile in that it would necessarily involve 
semantic arguments concerning the definitions of practically 
all substantive teras introduced fef.,° Black,  1973}< 
Instead, the discussion will focus on the type of role 
cognitive structure is likely to play in any explanation of 
human thinking. 

First, the notion of an internal structure places 
the discussion sguarely in the cognitivist rather than the 
behaviourist school of psychology. Cognitive structure has 
been described (Morgan, 1972) as an intervening variable 
which acts to form an individual's response to his 
environment. It is posited to be a result of experience, and 
to grow in the course of normal human development in 
organization and differentiation. Experience acts on 
cognitive structure both to add to it {in the sense of 
adding new experience) and to change it (in the sense of 
changing perceptions of and interrelations among old 
experiences). These two results of experience correspond 


closely to Piaget's (Inhelder and Piaget, 1958) assimilation 
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and accommodation. According to this line of reasoning, 
cognitive structure relating relevant concepts should play a 
role in determining response to the environment, and 
therefore differences in cognitive structure should evidence 
themselves in differences in response to the environment. 
The significance of a clear definition of cognitive 
structure to the justification of the study rests on the 
fact that the analytic technique of multidimensional scaling 
always produces a result. The problem of interpreting this 
result rests with the (at least partly) subjective 
interpretive abilities of the investigator. Cognitive 
structures which are the result of such interpretations may 
be more or less meaningless because of: {a) random error; 
{b) inappropriateness of the task to the age level tested; 
({c) inappropriateness of the assumptions of the analytic 
techniques to the data; and (da), inappropriate criteria for 
judging the reliability of the results. Any combination of 
the possibilities listed could lead to misinterpretation of 
results. Because of this possibility, the uncovered 
structures are best interpreted as hypotheses. Such 
hypotheses must pass the test of independent corroboration. 
This study has attempted to provide such corroboration, by 
endeavouring to relate differences in perception of concept 
relationships among groups of subjects to differences among 
groups on the personality variables. A major criterion for 
the meaningfulness of the results of the present study 


involves relating, ina theoretically satisfactory manner, 
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differences in subgroups! configurations to differences in 


their personality variables. 


An assumption must be made in order for research on 
concepts to proceed. An experimenter does not have direct 
access to concepts, in that concepts are not observable. 
They must be inferred on the basis of the observable 
behaviour (either verbal or otherwise) of the subject. There 
must be an assumption that the words (in verbal behaviour) 
as used by one subject share some presumably connotative 
commonality of meaning with the same words as used by 
another subject. That is, the word meanings, and therefore 
the concepts, are shared. Such an assumption does not mean 
that the meanings of terms as used by individuals must be 
coextensive. If such were true, there would be no reason for 
doing research on concepts. Introspection by the researcher 
would do as well as research. Nc commonality of word meaning 
makes the research impossible; perfect commonality of word 
meaning makes the research unnecessary. Kaplan (1964) points 
out that the meaning of theoretical expressions is always 
given in terms of a theory, that is, by using other 
theoretical expressions. In the language of this study, a 
concept is defined in terms of other concepts within the 
same cognitive structure. A shift in a subject's meaning for 
a concept implies a shift in the relationships among 
concepts, that is, a change in the cognitive structure. Only 


some commonality of meaning (i.e., “balipark commonality”) 
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is necessary for the research to proceed. From Kaplan's 
point of view, what we are investigating when we look at 
cognitive structure are the subtle shifts in meaning of 
concepts as perceived by different individuals. 

1.54 Construct Validity 

As mentioned above, the approach taken in this study 
fulfils one of the requirements for establishing construct 
validity given by Campbell and Fiske (1959) but not the 
other. If sets of similarity data on the same concepts 
gathered from the same subjects by quite different methods 
all indicate closely related results when analysed, this is 
evidence that the methods of finding similarity information 
converge on the same cognitive structure. Further, if a set 
of similarities when analysed by two different scaling 
techniques also converges on the same cognitive structure, 
this is demonstration that the convergence is not a function 
of the scaling method alone. In using both the above 
procedures, this research design conforms to the multimethod 
part of the Campbell and Fiske recommendations. 

With respect to discriminant validity, there does 
not seem to be a clearcut method of establishing validity 
when the construct under investigation is cognitive 
structure. For example, it seems futile to collect data ona 
second set of concepts. There is no parallel between showing 
that, say, a test of compulsiveness correlates poorly with a 
test of anxiety, and showing that a second set of concepts 


do not produce the same set of relationships among 
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themselves as the set of concepts under study. A possibly 
fruitful approach to establishing discriminant validity, and 
the one used in this study, involves collecting similarity 
data from different groups of subjects who might be 
hypothesized as having different cognitive structures and 
showing that this is indeed the case. 

Unfortunately, with concepts that are intended to 
have some significance in the domain of public education, 
such as the ones chosen for this study, the subject is not 
totally free to produce a completely unique set of 
Similarities. Kass (1971) found a strong tendency on the 
part of subjects to deviate very little from the group 
average in perception of a set of heavily content oriented 
mechanics concepts. In other words, the content structure 
inherent in the concepts obliterated individual differences. 
The subjects perceived that there were right answers. 
Carroll and Chang (1970), using political concepts for which 
there were no correct answers found good variation in 
perception. The concepts used in this study, while not as 
bound to content as those used by Kass or Shavelson C1972); 
are not as free to be individual as those of Carroll and 
Chang. Thus, multitrait validity can only be expected to 
appear as deviaticns from the "right answer" content 
structure, rather than as a quite unique perspective on a 
set of concepts. As will be seen below, the issue is even 
more complicated due to the lack of a statistical criterion 


for the measure-of-goodness of fit between structures used 
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in this study, namely, the non-metric version of the 
orthogonal Procrustes rotation (Lingoes and Schonemann, 
1974; Schonemann and Carroll, 1970). This technique will be 


referred to as the non-metric fit procedure. 


1.6 Procedure 

Six tests, three of concept similarity and three of 
personality variables, were administered to 132 subjects in 
grade 9 (N=78) and grade 12 (N=54). The concept similarity 
tests have been briefly described above. The personality 
tests were a test of develcpmental level, a test of 
cognitive complexity, and a test of field dependence- 
independence. The concept Similarity tests were analysed to 
produce matrices of similarities, which were converted to 
multidimensional configurations by the scaling techniques 
outlined above. 

Construct validity of the uncovered configurations 
was established primarily by examining the correspondence of 
the configurations to each other. The extent of 
correspondence was assessed by the technique of nonmetric 
orthogonal Procrustes rotation {the non-metric fit 
procedure). Graphical techniques were used to present a 
possible interpretation of this measure of fit. 

The subjects were assigned to groups according to 
factor loadings obtained from the Tucker analysis, and the 
Saliences on each dimension for the Carroll analysis. These 


subgroups were compared among each other and with the full 
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group for differences in cognitive structure, and for 
differences in the personality variables. Personality 
variables examined were 1.0. (grade nine only), achievement, 
developmental level, analytic-synthetic cognitive style 
(Witkin, 1965), and cognitive complexity (Seaman and Koenig, 
1974) as well as estimates of the subjects! personal 
consistencies on each measure, their total number of 
responses on the word association test, and an estimate of 
the individual overall reliabilities on the three measures 


for a given subject. 


1.7 Hypotheses 
The hypotheses of the study may be stated in summary 

form as follows: 
HO0(1). There will be no significant differences among the 
structural patterns revealed by Kruskal analysis of data 
based upon constrained association, similarity judgment, and 
semantic differential tests; 
HO(2). There will be no significant difference between the 
structural patterns uncovered within each grade among groups 
formed by the sequences of: 

(a) constrained association followed by Tucker 

analysis; 

(>) constrained association followed by Carroll 

analysis; 

(c) similarity judgment followed by Tucker analysis; 


(d) similarity judgment followed by Carroll analysis; 
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(e) semantic differential followed by Tucker analysis; 

(f) semantic differential followed by Carroll analysis. 
Pending the results of H0(2), there is a further hypothesis: 
H1(2). Differences in structural patterns as found in the 
groups as formed by the Tucker analysis, and again by the 
Carroll analysis, will be related to personality variables 


of the subjects. 


1.8 Summary 

This chapter has outlined the purpose and 
Significance of the study. Major terms have been clarified. 
The problem to be addressed has been stated, a procedure for 
doing so has been outlined, and the problem has been put in 
the form of testable hypotheses. 

Chapter 2 of this report summarizes literature 
related to the study. Chapter 3 presents details of the 
administration of the study, the scoring procedures, and the 
analytic procedures. Chapter 4 contains results and 
discussion, while Chapter 5 summarizes and concludes’ the 


report. 
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CHAPTER 2 


REVIEW OF RELATED LITERATURE 


2-1 Introduction 

This chapter is divided into four main sections. 
Section 2.2 discusses research focused on measuring concept 
relationships in an educational context, with particular 
attention to science education. Section 2.3 discusses the 
rationale and assumptions of the three multidimensional 
scaling procedures used in the study. Examples of the uses 
of these techniques in the literature are briefly outlined. 
The next section deals with the origins and uses of the 
three similarity rating tasks. Section 2.5 concerns the 
three personality variables tasks, as well as a brief 
mention of other personality variables generated during the 
course of the analysis. Of necessity, the sections of this 
chapter refer both back and forth to each other toa 


considerable extent. 


2.2 Concept Relationship Research 

The purpose of this section is to trace the 
development of attempts to measure concept relationships, 
primarily in science education research. Studies. have been 


conducted on the relationship of cognitive structure to 
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achievement, the usefulness of word association tests in the 
uncovering of cognitive structure, and the relationship of 
cognitive structure to subject matter structure. Some of the 
research has used scaling techniques, and some of the 
research has attempted to measure individual differences in 
cognitive structure. All of the research has suffered from 
conceptual flaws and/or unavailability of appropriate 
analytic tools. 

Johnson (1964; 19653; 1967) conducted a series of 
studies showing the possibilities for uncovering similarity 
data from verbal association tests, for relating verbal 
output on such tests to achievement, and for relating 
Similarity matrices produced by the above method to those 
produced by similarity judgment techniques. In the first 
study (1964) he compared the strengths of associative 
meanings among physics concepts for four groups of subjects 
(50-70 per group) with varying degrees of involvement with 
physics. The groups were all grade 11 and 12 giris: (1) 
having taken physics; (2) taking physics; (3) planning to 
take physics; and (4) not planning to take physics. Subjects 
responded with one written free association to each of a 
series of 18 concepts read aloud 5 seconds apart. The 
frequencies with which the 18 concepts were used as 
responses to each concept were tabulated and converted to a 
matrix of percentage similarity. Significant differences in 
the average similarity among all four groups were found in 


the order 2, 1, 3, 4. In the second study (1965), Johnson, 
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using as subjects 166 female high school seniors taking 
physics, gave a written constrained version of the word 
association test along with a ten-problem physics test. Half 
of the sample took the tests in the reverse order from the 
other half. Subjects were constrained to write the first 
physics word which occurred to them as a response. Responses 
were categorized as (a) words from the concept list, and (b) 
"responses which occurred together with their respective 
stimuli in a constraint necessary for the solution of one of 
the ten problems." Concepts were chosen so that, for all 
cases, category (b) was included in category (a). Subjects 
who had the problem test first produced significantly more 
responses of both types {a) and {(b). Subjects who had the 
association test first scored significantly higher on the 
problem test. The correlation between number of (a) 
responses and test achievement was 0.34 =(p<.05) for the 
group which had the association test first, but -0.03 for 
the other group. For type (b) responses, the correlation was 
0.42 (p<.05) for the association test first group, and 0.19 
for the other group. Johnson also reported many results for 
+he individual concepts. He concluded "language habits which 
have been learned among words in physics are an integral 
part of the conceptual framework a student utilizes in 
dealing with concepts in the subject matter". 

An important limitation to the work of Johnson up to 
the time of his study was the fact that his analytic 


technique could not make use of responses common to two 
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stimuli but not on the list of concepts to increase the 
degree of calculated association. Such a problem is 
circumvented by the Garskof and Houston (1963) method, 
decribed in detail below. 

In a third study, Johnson (1967) used the 
calculation procedure of Garskof and Houston along with a 
Similarity judgment test on fourteen physics concepts using 
a sample of 24 high school seniors (16 male, 8 female) 
divided into 12 high and 12 low achievers. The concepts were 
ranked according to their occurrence in subject matter 
presentation (presumably by a textbook word count, but this 
is not reported) and according to their frequency of 
occurrence aS responses by the two groups. Rank order 
correlations between the orders produced by each group and 
that of the subject matter were 0.87 (p<.001) for the high 
group and 0.52 (p>.05) for the low group. When the order of 
occurrence for each concept response within the response 
list produced was taken into account, the respective 
correlations became 0.52 (p>.05) and -0.05 (p>.05). Johnson 
defined a constrained association (in a manner not to be 
confused with the use of the term in this report) as a 
relation constrained by a law of physics. For example, 
"force"®, "mnass”, and "acceleration" are constrained 
responses to each other because of the physical law "force = 
mass x acceleration". High achievers gave significantly more 
constrained associations than low achievers, both 


considering and not considering order of response. After 
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calculating Garskof and Houston relatedness indices for both 
groups, and similarity judgment indices based upon that part 
of the experiment, Johnson reported a rank order correlation 
between the two of 0.75 (p<.001) for the high group and 0.65 
(p<.001) for the low group. The significance of Johnson's 
work for the present study lies in the estimation of the 
bhagnitude of the correlation between the similarities 
gathered by the two techniques. 

In a follow up study, Johnson, Curran and Cox (1971) 
gave a free association test, a constrained association 
test, and a similarity rating test to a group of 49 male 
physics majors who had completed at least three years of 
college. The nature of the constraint on the constrained 
associaticn test was that subjects were asked to list the 
words they would use in defining the stimulus word. The 
authors report good correspondence for both the free and 
constrained association tests with their hypothesized degree 
of relationship among the nine stimulus concepts. Of more 
interest here are the rank order correlations among the 
indices of relationship produced by the three methods. These 
are: free and constrained association, 0.90% free 
association and similarity judgment, 0.85; and constrained 
association and similarity judgment, 0.79. These results 
lend credence to the expectation that correspondence among 
the various methods should be expected in the present study. 
Such correspondences probably should not be as large as 


Johnson's results because of the highly homogeneous nature 
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of the subjects he used compared with those in the present 
study. 

Reporting further analysis of what appears to be the 
same data, but using only six of the concepts, Johnson, Cox, 
and Curran (1970) performed a Kruskal scaling on the six 
points using the free association data and the similarity 
rating data. Relatedness indices were calculated as in 
Johnson's earlier work rather than by the Garskof and 
Houston method. Despite citing Xlahr's (1969) caution 
against using such a small number of points in a scaling 
analysis, Johnson et al reported solutions in two and three 
dimensions. Although they achieved zero stress in three 
dimensions, they admit that there is a possibility of such 
an occurrence by chance. Scaling of such a small number of 
points appears to be a misuse of the Kruskal technique. 

Evanechko and Maguire (1972) have studied the 
dimensionality of children's meaning space. Although this 
study was not related to science education, it is included 
in this review because of the comparability of both the 
techniques and the ages of the subjects to the present 
study. A Semantic Features Test was developed to investigate 
the types of definitions preferred by children. Evanechko 
and Maguire produced a list of 24 ways of perceiving meaning 
between words. Examples are: synonym, attribute, whole-part, 
generic definition, etc. Fach of the 24 categories was used 
to prepare 23 example pairs of the particular definitional 


or meaning relationship. For example, under "whole-part" 
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were bird-wing, hand-finger, and 21 others. Using this 
technique, it was possible to produce 276 possible pairs of 
the 24 categories without repeating the actual examples. The 
Subjects were presented with the 276 pairs of pairs and 
asked to decide which pair constituted the better definition 
of the first word in each pair by the second. The directions 
given the subjects were to "think of the kind of meaning 
given for each word and choose the one which is nearer to 
what you think the word means in each pair of statements". 
The sample consisted of 266 students in the fifth 
and eighth grades. A group average proportion of times one 
meaning category was preferred over another was calculated 
for the pairs of categories, producing a 24 x 24 matrix of 
proportions of preference. These values were deviated around 
0.50, and the absolute values taken to produce a matrix of 
dissimilarities. This matrix was scaled by the Kruskal 
technique. Four dimensional solutions for each grade were 
reported, with stress 1 (variance unaccounted for) close to 
16 percent in both cases. For purposes of interpretation, 
the 24 categories of meaning were grouped into five larger 
categories. Reasonable interpretations of the four 
dimensions were given for both grade levels. An orthogonal 
Procrustes rotation (Schonemann, 1966) of the grade eight 
solution to the grade five solution indicated a poor fit. 
Since the metric version of the rotation was used, the 
conclusion presented by Evanechko and Maguire that the two 


solutions are substantially different must be doubted. 
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Kruskal analysis is a technique designed to apply to ordinal 
data. The orthogonal Procrustes rotation, designed for at 
least interval data, is not appropriate. It seems a rather 
large leap from the directions given to the subjects to the 
reporting of the results in which the terms "meaning" and 
"relationship" seem to be almost synonymous. Further doubt 
must be cast upon the results because of the taking of the 
absolute values of the differences between the proportions 
of preference and the value 0.50. By this method, the 
proportions 0.30 and 0.70 become identical. While the 
"dissimilarity" has been preserved, the "preference" has 
been lost. The study provides useful information about the 
Similarity of the relationship categories as perceived by 
the subjects, but no information about their preferences. 
The significance of the Evanechko and Maguire study is that 
it demonstrates that, on group average data, and despite 
serious conceptual flaws, subjects as young as grade five 
will produce meaningful resuits on a task which is 
comparable in difficulty to the tasks in this study. 

Kass (1971), using a sample of 353 grade 12 physics 
students, obtained similarity ratings on the 190 possible 
pairs of 20 mechanics concepts. Her concepts were described 
by short phrases and/or formulae rather than merely being 
named. Each subject was given the pairs of stimuli on 190 
cards and told to rate their similarity in difficulty ona 
scale of 1to 9. This technique corresponds most closely 


with the similarity judgment technique of the present study, 
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except for the imposition of a definition of similarity in 
terms of difficulty. Such an imposition may be expected to 
distort the similarity judgments of the subjects, for it is 
possible that a subject might consider a pair of otherwise 
unrelated concepts as being quite similar in difficulty but 
in no other aspects. Kass analysed the resulting matrix of 
Similarities by the Tucker-Messick method, but found that 
there were insufficient differences in individual 
perceptions to warrant pursuit of the individual aspects of 
the study. As mentioned above she suggested that the use of 
heavily content oriented concepts, taken directly from 
course content may reflect mostly the structure of the 
discipline, as viewed by subject matter experts, and very 
little of the individual perceptions of the subjects. Using 
group average data, three stratified (by class) random 
samples of 60 subjects each were compared in group average 
cognitive structure. Using Kruskal's stress 1 and 
interpretability of the loadings in terms of concept meaning 
as criteria, the four and five dimensional solutions were 
judged most satisfactory. Stress 1 values were between seven 
and ten percent. Comparisons of the loadings was done by the 
use of the Kaiser (1960) factor match test, a metric 
technique. The non-metric version of the orthogonal 
Procrustes technique was unavailable at the time. No 
goodness of fit measure between the structures produced by 
the two groups is reported. In the cases of both the four 


and five dimensional solutions, Kass offered reasonable 


|, Midenesane, dyno: nant vandllietan* & 


aa i oa 


~ 


rater 

ni ¥4¢aetiate we soda teh S to potrinoant oseh ag J 
: sth QC pairs 

“* fh « @rpeeyx 9 ot 2 oh act ivkpodad ns done pe DS oo 


<j} +P apt yetoerdue eat Vo miaaephor corse itate = 7 | 
oereyedito io . "460 "8 sahfenon ¢tdpia toordar 6 + ssi 


red veto? 7 


to ortse ealtinaer sh feng isns 2287 220806 cadens i 
toed Rapat ted ,hodtee aokeeet -tetouT odd yd ROESET 
reo icky Bhat Lo essreTeR?2h «© tneboitiwedh (oTow en 
10 Sinedhs Panhivibnt od P to tivewq Joa ttSyv of anoitq9oT 


ees 
1s 6C ey 6 Odt te botreppoe ata ovods beso ite ek .ybuta ode 7 


fenazvih tacet  cetqennoo ~hetnoiio taesao5 yi 


me 7 os 
: ! weptont+s 4&9 yfteoe toelies Yyse s%ae7009 927009 - 
veov ba)  .etreexe wdtdoem dooftdve yd fewreivy 26 oniigionts 
f .o*oerde2 af? Toa nae RIGS0789 teubivibal off To- ottret 4 
(jaels we) hesdEae 24s eerdd ,«htseb spRpToVs eagll 
opr ave “ow wi Retedtaen eteav dose atootdua 04 so.netaden 
Bae 1 noi *s = Posner part! ot tous Te ov itinpor : 
OPiny @ taecsoo Yo aaaet ab epatfeol edt Jo yiilideserg o: ‘k 7 
$ snottyton Leaotenemth ovit hoe wot edt sie “a 


neves geoewted otew eepulhev ul aneite. .ytotoslaitnse teoa bot 3 


aa io 

sit yc one saw apathsol wks to enoaiteqaod .meoTeq ot bos 
. 

Shrtod) ® ¢teet, dosew TOeIST (HORT), ssekee ae Pace 


a 
Bb: eedaze ant 19 sokatevy okatom-non oar. ie 
tc 
ww «sesmt+ ede ts oldslisyvsnu 25 supindoas oo 
va Lact hee eertutoutte aft asovted alvesom 482 to mam ronbo 


-” 


tu07 ed? dtod to asesn oft at eae ai Faverto 
oe : 


31 


explanations of the loadings of the concepts on the factors 
in terms of the concept meanings for three of the 
dimensions. 

Shavelson has published a series of studies 
investigating the relationship between content structure as 
represented in instruction, and cognitive structure as 
represented in student memory (Shavelson, 1972, 1973, 1974; 
Shavelson and Stanton, 1975) In the first study, he employed 
a combination of grammatical analysis and digraph theory 
(Harary, Norman, and Cartwright, 1965) to recover a matrix 
of similarities among physics concepts from the students! 
text. Digraph (directed graph) theory is a mathematical 
technique in which concepts are represented by points, and 
the relationships among them by directed lines joining the 
points. Harary, Norman, and Cartwright provide a discussion 
of the abstract properties of the networks drawn, but do not 
concern themselves with the details of correlating 
grammatical analysis with their system. Shavelson (1972) 
gives some clues to the technique, using the text sentence 
"Force is the product of mass and acceleration” to produce a 
graph linking the concepts "force", "product", "mass", and 
"acceleration" with the relational properties “is", "of", 
and "and", The technique is not completely clear from one 
example, and Shavelson refers the reader to Shavelson 
(1970), which is not readily available, for further details. 
Some further detail is provided by Shavelson and Geeslin 


(1975), but not enough to allow replication of the method. 
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Harary, Norman, and Cartwright provide details of a method 
for linking digraphs containing common elements to each 
other, and for converting this large digraph to a matrix of 
dissimilarities. Shavelson analysed a physics text, and 
produced a digraph for each sentence of the text which 
contained two or more of his list of 14 mechanics concepts. 
Since the distance between two points in a digrabdh is the 
number of directed lines between the points, Shavelson 
eliminated all but those digranvhs which produced the 
shortest path between any two concepts. He then combined 
digraphs and produced a matrix of subject matter 
dissimilarities. Shavelscn interpreted this matrix as 
representing the content structure of the material. 
Cognitive structure in the 1972 study was estimated 
by an unconstrained free association test using the fourteen 
concepts as stimuli. Relatedness indices were calculated by 
the Garskof and Houston method. Using a sample of 40 naive 
but interested high school age volunteers (28 experimental 
and 12 control), Shavelson measured changes in cognitive 
structure over five days of instruction based on the text 
which produced the content structure described above. A 
Significant gain on an achievement test and an insignificant 
increase over time in the total number of responses provided 
by the subjects to the association test are reported. In 
addition, subjects in the experimental group had a median 
relatedness index which increased regularly across the six 


administrations of the test (an initial test and one 
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following each day of instruction), while that of the 
control group remained low and stable. The control group 
completed all the tests of the experimental group, but spent 
the rest of the treatment time participating in another 
(unreported) study. 

Shavelson produced a matrix of Fuclidean distances 
between all pairs of relatedness index matrices. This was a 
Six days by six days matrix of distances between relatedness 
index matrices. The method involves {a) subtracting the 
corresponding values for all relatedness indices in the two 
matrices, (b) squaring and summing these values, and (c), 
dividing by the number of entries in a matrix and taking the 
Square root of the result. For the control group, the 
average distances were uniform and small over the six days, 
but for the experimental group, the average distance was 
fairly regularly related to the separation in time of the 
testing occasions. Shavelson did Similar calculations 
comparing the control group and experimental group matrices 
on the same day. Similar results allowed him tc conclude 
that the two groups started out with similar cognitive 
structures, but that that of the experimental group grew 
away from that of the control group. 

These analyses were followed by conventional Kruskal 
scaling on the matrices produced by each group each day. 
Using the two dimensional solutions, which produced stress 1 
(presumably) less than 10 percent, Shavelson could not 


corroborate the conclusions based upon his average distance 
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calculations, and had to resort to rather weak explanations 
involving the control group having "prescientific" meanings 
for the concepts which corresponded quite closely to the 
scientific meanings. Shavelson performed his average 
distance comparison of the cognitive structure with the 
content structure revealed by the digraph analysis. This 
calculation showed that the experimental group grew towards 
the content structure, while the control group did not. 
Again, Kruskal analysis of these structures failed to 
confirm these conclusions. 

Shavelson (1973) has reported further results on the 
Same experiment. On the word association test, if only 
responses from the original stimulus list are counted 
(constrained responses, in Shavelson's terminology), then 
there is a significant relationship between number of 
responses and achievement on the achievement test, 
especially when the constrained response occurred early in 
the list of responses. Shavelson reports a significant but 
small relationship between achievement and the Hidden 
Figures Test (French et al, 1965), which is used in the 
present study. Unfortunately, he does not report the results 
relating the HFT to differences in cognitive structure. In a 
longer article (1974) which mentions the earlier results, 
Shavelson maintained that multidimensional scaling of a 
matrix of relatedness indices taps cognitive structure. As 
an alternative method of collecting information on cognitive 


structure, Shavelson suggested lexical graphing methods 
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(Rapoport et al , 1966). This suggestion has not been used 
in the present study because of the complexity of the 
directions needed for a subject to perform a lexical 
graphing. The technique has possibilities, however, for work 
with more mature subjects. 

Shavelson's data raise a lot of questions. Preece 
(1976c) has responded ina note to Shavelson (1972, 1973, 
1974) regarding the use of his average distance between 
matrices calculation. Preece demonstrated by a neat example 
that the technique is inappropriate, in that the value 
depends as much upon the average absolute value of the 
relatedness indices as upon any change in their relative 
Sizes. On the basis of this criticism, and his own scaling 
results, Shavelson's data do not support his conclusion that 
cognitive structure "grows towards" content structure with 
instruction. The idea is intuitively appealing, however, and 
it would he interesting to attempt to use the goodness-of- 
fit measure of the nonmetric orthogonal Procrustes rotation 
as a measure of growth on his data. It would be a reasonable 
hypothesis that, as the experimental group progressed 
through instruction, their geometric representation of the 
concepts, as found by Kruskal analysis, would yield 
successively better measures of fit when rotated to the 
content structure as revealed by digraph analysis. 
Conversely, similar results for the control group should not 
yield improving goodness-of-fit measures. 


Shavelson and Stanton (1975) seem to view the 
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convergence cf different methods of measuring cognitive 
structure on the same similarity matrix as validation for 
the construct “cognitive structure". The methods under 
discussion were: word association, card sorting, and lexical 
graph building. This suggestion is of relevance to the 
current study because of the similarity in intent and 
underlying assumptions of the card sort technique to the 
Similarity judgment technique (Torgerson, 1958) as used in 
the present study. 

Preece {1976a, 1976b) has reported two studies of 
interest. in the first study, he measured the relationships 
among 15 mechanics concepts using a sample of 28 physical 
science student teachers, all with degrees in science. 
Subjects were presented a written free association test, a 
controlled associaticn test, and a tree ccnstruction test 
Similar to the lexical graphing method of Rapoport et al 
(1966). In a tree construction task, subjects are presented 
with a list cf concepts and are asked to select the two most 
closely related. They write this pair in the centre of a 
page and join them by a line. They then choose a concept 
from the remaining list that is most clesely related to 
either of the concepts already chosen and add it to the 
page, joining it by a line to the concept it is most closely 
related to. The actual directions are more complex, but 
essentially the subject continues in this manner until the 
list of concepts has been exhausted. The control in the 


controlled associaticn test asked subjects to list physics 
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words as responses, and asked for exactly five responses. 
Preece obtained what he described as very poor relationships 
between the results of the association methods and the tree 
method. This judgment by Preece of his own data seems harsh. 
Preece produced six matrices of similarity from the 
responses of each subject. Four of these were based upon 
association tests and two on tree construction tests. On 
group average data, and multiplying by minus one where 
necessary to compare similarities and dissimilarities, the 
lowest correlation between any pair of tests was 0.8 
(p<.001). When compared with his theoretical prediction, the 
lowest correlation found is 0.60 (p<.001). 

The tree method is considered appropriate when a 
geometrical model will not fit the data. Such a situation 
would occur if the concepts grouped themselves into closely 
related clusters separated by relatively large distances. 
Under such conditions, scaling techniques would produce a 
degenerate solution, with large distances exaggerated and 
small ones attenuated. Such conditions were not a problem in 
either Preece’s study or the present one. It is important to 
note that the type of association test used in the present 
study is closer in spirit to Preece's free association test 
than to his controlled association test. On the free 
association, Preece reports one month test-retest 
reliability of 0.86 using eighteen year old subjects. This 
reliability appears to have been based on the second study 


(Preece, 1976b), in which he used the Carroll scaling method 
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on a sample of boys ages 11, 14, and 18 from a boy's grammar 
school. The boys responded to the same concepts as the 
student teachers. Details in this second report are sketchy. 
There evidently were individual differences among subjects, 
but we are given no indication of their magnitude or their 
nature in terms of either the subjects or the concepts. 

It appears that Preece misinterpreted the plot of 
subjects in dimension space produced by the Carroll progran. 
He interpreted a subject's distance from the origin as a 
reliability, in the sense that it measures conformity of the 
subject's actual dissimilarities with the subject's 
estimated distances. While the measure does indeed give a 
correlation between a subject's actual data and his 
estimated data, this correlation is not directly 
interpretable as a reliability of either the subject's 
responses or of the technique as applied to his data. The 
subject's conformity to the group average will largely 
determine the variance in his responses which can be 
accounted for by any weighted composite of the group average 
perception space. For interpretation of an individual's 
loadings in a Carroll analysis, there are two "best fit" 
steps. One involves fitting the group average similarity 
data into concept space, and the second involves fitting the 
individuals into subject space. The resulting distance of a 
subject from the origin of subject space depends upon the 
error in the group average data, the goodness-of-fit of the 


group average solution, the homogeneity of the group, the 
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conformity of the subject to the group average, and the 
error in the subject's data. Direct interpretation of this 
distance from the origin is not recommended. 

In summary, Johnson established the possibility of 
using word association tests to measure concept 
relationships. He also linked achievement to responses on 
association tests, and presented evidence of convergence 
between association tests and similarity judgment tests. 
Evanechko and Maguire showed that similarity judgment 
methods would yield reliable results with subjects even 
younger than those in the present study. Kass made the first 
attempt at finding individual differences in cognitive 
structure, but found that heavily content oriented concepts 
yielded only small variation among subjects. Shavelson, 
although he used methods open to criticism, attempted to 
measure cognitive growth in terms of changing cognitive 
structures. He showed good convergence between similarity 
judgment and word association methods. Preece presented 
sketchy results on the use of the Carroll method, but gave 
highly encouraging reliability estimates for the association 
test. 

None of the literature cited above report the use of 
the non-metric measure of goodness-of-fit used in the 
present study. Further, the use of the semantic differential 
technique in comparison with the other two methods of 
uncovering structure appears to be unigue in science 


education research to this study. 
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2.3 Multidimensional Scaling 

The purpose of this section is to outline the 
differences in assumptions and applications of the three 
types of scaling procedures used in this study. These are: 
multidimensional scaling (Kruskal, 1964a, 1964b; Kruskal and 
Carmone, 1971); points of view analysis (Tucker and Messick, 
1963); and individual differences analysis (Carroll and 
Chang, .1970). Subkoviak (1975) has produced a substantial 
review of the use of these techniques in educational 
research. 

Initiating the work later extended by Kruskal, 
Shepard (1962a, 1962b) devised a technique to find the 
minimum number of dimensions necessary in which to portray a 
set of n points such that the computed distances between all 
possible pairs of points are monotonically related to the 
set of dissimilarities of the original data. The usefulness 
of this technique depends upon the assumption that monotonic 
constraints imposed in sufficient number on a set of points 
will approximate the interpoint distances to within a linear 
transformation of the dissimilarities. 

Shepard has achieved good results with this model. 
tnvcan artificial example, a set of 15 random points were 
generated in two dimensions. A set of interpoint distances 
were calculated, and these served as dissimilarities. The 
actual coordinates of the points were then ignored in the 
analysis. Shepard produced a set of points in two dimensions 


quite close to the original points. 
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In a second experiment, using data gathered from 
people, his subjects were asked to rate the degree of 
Similarity of facial expressions ina series of photographs 
of an actress. The actress had been given specific 
instructions for each facial expression. The task asked the 
subjects to give numerical ratings to the difference in 
facial expression for all pairs of photographs. These 
ratings, however, were used only as rank orders, not as 
interval data. The results of the Shepard analysis yielded 
two dimensions, a horizontal dimension of pleasantness, and 
a vertical dimension of degree of emotional arousal. The 
results agree closely with the directions given to the 
actress and to the results of an earlier analysis, done in a 
different manner. This original analysis imposed much 
stricter assumptions on the data than those imposed by 
Shepard. 

In a third example involving similarity ratings of 
colours the analysis produced a two dimensional circular 
pattern corresponding very closely to a circular spectrun, 
with the red and violet ends meeting. 

Kruskal (1964a, 1964b), working with Shepard, 
improved the model by producing a theoretically more 
rigourous method of computation and by providing a measure 
of goodness-of-fit of the solution obtained to the original 
data. His goodness-of-fit measure, stress, is analogous’ to 
{square root of) proportion of error variance in a 


regression analysis. The computational algorithm seeks to 
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minimize stress. The stress can be interpreted roughly as 
percentage disagreement between solution and data. The 
stress supplies an objective criterion for deciding when the 
best solution has been reached. Stress is a value which 
increases as the relation between dissimilarities and 
distances becomes poorer. One output of the Kruskal 
procedure is a plot of stress vs. the number of dimensions 
for the best possible solution in that number of dimensions. 
The stress increases as the number of dimensions goes down. 
If inclusion of an extra dimension significantly decreases 
the stress, then the dimension should be included. If not, 
then it should not. Kruskal gave verbal definitions of the 
quality of the results ranging from 20% ("poor") to 2.5% 
("excellent"). The quality of his results was similar to 
that of Shepard. 

Jackson and Messick (1963) have criticized the 
traditional multidimensional scaling procedures for failure 
to account for variance among subjects in the manner in 
which they perceive the stimuli under study. The Kruskal 
model is able to work only upon the average perceptions of 
the group of subjects. Jackson and Messick have argued that 
some subjects may be cognitively complex, and perceive the 
stimuli as varying in many dimensions, while other subjects 
may be cognitively simple, and view the same stimuli as 
varying along relatively few dimensions. They put forward 
the rationale, and in a companion paper, Tucker and Messick 


(1963) presented a mathematical method for accounting for 
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the variation in individual responses to a set of stimuli. 
Tucker and Messick proposed the use of principal components 
analysis on the matrix of sums of squares and cross products 
of a subjects by subjects matrix (formed from the subjects 
by similarities matrix of raw data) to attempt to describe 
the variance in the perception matrix by a smaller number of 
dimensions than the original number of subjects. Principal 
components analysis of the responses of the n subjects is 
carried out in order to represent the variation in the 
results in terms of a smaller number of basic subject 
"types". That is, if they had, for example, 50 subjects, the 
variance in the scores of the fifty subjects would yield, 
say, four factors. This result is interpreted by assuming 
that most of the fifty subjects fall into four types. After 
the principal components analysis, four "ideal individuals", 
either composites put together from each of the four groups, 
or four actual individuals who might be said to personify 
the types are identified. A Kruskal type scaling is then 
carried out for each of the ideal types. Fach of these 
scaling results would be based upon a much more homogeneous 
group of individuals than the scaling procedure applied to 
all fifty subjects. In such models, different types of 
idealized individuals yield different geometrical patterns 
of stimulus-point distances. Ideally, differences in 
patterns can be traced back to subject characteristics. It 
may be possible to show that, for example, a two dimensional 


pattern of distances can account for the responses of those 
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who have been characterized {by some independent measure) as 
cognitively simple, while, for example, a four dimensional 
pattern may be necessary for the cognitively complex. 

Jackson and Messick reported a Tucker and Messick 
study conducted earlier in which subjects were asked to rate 
preceived similarities between international political 
figures of the 1950's. The principal component analysis 
revealed the presence of three ideal types. The scaling 
pattern results for the three types were quite different. 
One type had a one-dimensional pattern, with a heavy cold- 
war evaluative emphasis. The second type revealed a two 
dimensional pattern, interpreted as a Democratic—-Republican 
dimension and an evaluative dimension. The third type had a 
seven dimensional pattern which defied complete 
interpretation, but that did reveal political nuances which 
the other ideal types had not revealed. This third group was 
regarded as the most cognitively complex. Tucker analysis 
has also been used by Kass (1971), as described above. 

In the present study, the Tucker technique has the 
potential to identify different types of individuals. In the 
principal components analysis, those subjects whose views of 
the concepts are most similar should load strongly on the 
sane dimensions. If those who load strongly on each 
dimension are taken as a group, it is possible to analyse 
the perceptions of each of these groups in order to identify 
differences in perception of the concepts among the groups. 


Further, it is also possible to analyse the personality 


an (steo Kon deisel deninis ence YA) berkissas sees abow over ow 
fennianetih ‘wot 6 .obeaxe jor ,orthe ,olow ba nfov isbayes 
elon yievininges (ait tet eire2e%oe ad yao azettag 


{steaee? hae seeor” Ff bate 4a treet has uenados h 
stoofdpe Hoidw al retiree betoohaos ybuta 


sie7 of botee OTF" 
fenitited feaodtanretad qoeewind eeitital inte hevieoeig 
‘toyieag *auNoEe TOD Lnqiontoq aa? .2torrt od? Yo eompi® 
onifeoe aft .Beny? feabi oatd? %O soneeedq od? hofrovot 
toarelti® ot¢iup etew aeqy* ee7d? ad to? wtfuses metteq 
-hiem weed & dew aaes sen fpacianssth-eao 6 bed oay* end 
owt © heleovet eaqyt ‘haotean  oAT atendaee avitertsve Tew 
geottdudet-ol* eae oeget £ Sh fotetatetai ares teq Ie aoftenoedh 
s ted Hoo? Beha? af? ftotenerih oviseulsve ap bie aolanomib 
ete tas hot toh dorde azs+tnd {snoianeaib nevee 
intdy mooeson Deoktifog Iseved fib ted? tod ,noitetoaqtetat 
jy sence brid? abd? .bebesvet ton hed zeqy? Teoht yoAdt0 Od? 
2i2visae ywoton? .xatgqaod ylovitinpo> tzeon od? a6 bebrapes 

.evolln Aodtizozeh 26 ,¢Ptet) east yd boas neud oels asd 
”* ond ouptadoes weitnvt? off vainee taesaciq oft at _ 
o44 wT .tfoubivebat to peayd Saentettib ytitaohl oF {se itnotog 
'y e¥edv eaode atpoopdre anod? yetayinne s*aerogno Laqtontaq 
“+ o vipsette feol KieGds aliet= +208 e165 efgoonoo 689 
ines no yieoorts beol ow esodt 1 .enotvaqmih sonne 
guyttme of @fdienoq st #2 -yauo7p & es edn? Ob ao tenemib 
e Pi toehi of tebre ai eapotp efed? © dose to iment 
.agnayy aft paoes etqscnoo edt Fo aoktqeo7eq mk’ 


vtileaoetoe St? oayienn of Sfdienog oplp et 


45 


variables of the members of each of these groups in order to 
identify differences among groups on the personality 
variables. While the Tucker analysis cannot, by itself, 
separate complex from simple patterns of perception, 
subsequent Kruskal analysis of the perceptions within each 
group can. 

Carroll and Chang (1970) and Carroll (1972) have 
developed a scaling method based upon entirely different 
assumptions from those of Tucker and Messick. Their method 
assumes that the same dimensions exist in “psychological 
Space" for all subjects, but that the importance or 
“salience” of these dimensions can vary. The result of their 
analysis produces a stimulus space plot and a dimension 
space plot. The stimulus space plot is similar to the 
results that one would obtain from a Kruskal analysis of 
group data. The dimension space, however, represents the 
subjects* positions with respect to the importance which 
they attach to the dimensions of the group analysis. An 
example will clarify the nature of the Carroll and Chang 
analysis. In a classic study, (see Fig. 1) subjects were 
asked to rate the similarity of a group of twenty countries. 
Group analysis revealed a dimension which could be 
interpreted as developed- underdeveloped, and another 
dimension which could be interpreted as Communist-Western. 
Subjects were classified independently as doves and hawks. 
The dimension space plot (upper right) shows that the hawks 


gave much more salience to the political dimension, while 
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the doves gave more salience to the economic dimension. As a 
result, the shape of the plot of nations is stretched or 
shrunken along one dimension or the other. As can be seen in 
the diagram, Cuba and Brazil were rated as quite similar, 
because of the salience of the economic dimension, by the 
doves, (bottom right) but quite different, because of the 
political dimension , by the hawks (bottom left). 

In another example (see Fig. 2) rating similarities 
of colours, subjects rated as red-green colourblind (bottom) 
"shrank" the red-green dimension of the stimulus space, 
while those rated as yellow-blue colourblind (top) shrank 
the yellow-blue dimension. The Carroll and Chang method is a 
metric scaling technique. As such, it is open to criticism 
on the grounds that it imposes too strong conditions on the 
data. The elegance of the two examples quoted, however, 
testifies to the robustness of the method with respect to 
violation of the metric assumptions. 

In the present study, the Carroll method offers a 
technique for identifying individual differences in 
perception based upon quite different assumptions than the 
Tucker technique. The Carroll technique postulates the same 
common n-dimensional space for all subjects, but allows the 
salience of the dimensions +o vary among subjects. Thus, the 
technique can indicate directly those who share a particular 
view of the concepts under study. Not only is it possible, 
as with the Tucker technique, to identify those who share a 


common view, but with the Carroll technique it is also 
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An INbsc‘.. analysis of Helm’s 
data on color perception produced the 
group stimulus space* shown in (a) and 
the subject space shown in (b). The di- 
mensions are normalized so that the sum 
of squared coordinates on each dimen- 
sion = 1.00. The coding of the colors 
in the group stimulus space is as follows: 
Re= reds ¥o=— yellow: Gy (1)) = sreen 
yellow; GY(2) = green yellow with more {@} SUBJECT CD4a 
green than GY(1); G = green; B = blue; 

PB = purple blue; BG)s= purple: 

P(2) = purple with more red than P(1); 

RP = red purple. In the subject space, DIM 2 
CD1 through CD4 are four red-green 
color-deficient subjects (CD4a and CD4b 
are replications for one subject), while 
N1 through N1O are normal subjects 
(N10a and N10b are replications for one 
subject). Diagrams (c) through (f) are 
the private perceptual spaces for four 
subjects. 
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possible to identify the dimensions of concept similarity 
which are used by the subjects in each group to produce 
their similarity data. Further analysis by the Kruskal 
technique, according to Carroll and Chang, is unnecessary. 
Such analysis, however, was undertaken as part of the 
present study in order to validate the Carroll analysis 
findings. As mentioned above (P.38), the dimensions assigned 
to a subject by the Carroll technique may be conceived as 
incorporating two best fits, one of the solution to the 
data, and another of the individual to the group average. A 
subgroup average solution may be found either directly from 
the average saliences of the subgroup members on the group 
average Carroll solution, or by returning to the original 
data, forming a subgroup average matrix and performing a 
Kruskal calculation. Since the second method involves only 
one best fit, and is thus less removed from the data, it was 
adopted for the present study. 

The Carroll technique imposes metric assumptions on 
the data, while the Tucker technique, if used as Ross (1966) 
suggests {details given in Chapter 3), is non-metric. 
Convergent results based upon these two methods which differ 
in assumptions, calculational technigues, and nature of 
result are consistent with the Campbell and Fiske (1959) 


multimethod validity as applied to this study. 
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2.4 Similarity Rating Tests 

The above discussion focused upon the analytic 
techniques used to analyse the similarity matrices obtained 
from the concept similarity rating tests presented to the 
subjects. This section concentrates on the methods by which 
the concept similarity ratings were gathered from the 
subjects. These methods are: word association, similarity 


judgment, and semantic differential techniques. 


2-41 Constrained Association Test 

The Constrained Association test is based upon a 
method proposed by Garskof and Houston (1963). Repeated free 
associations based upon a set of stimulus words which were 
selected in pairs so that the pairs ranked along a continuum 
from “high related" to "low related" were given to a group 
of twenty undergraduates. Subjects were presented with 
Stimulus words, one to a page, and instructed in the 
continued free association technique. (The free association 
technique requires the subject to respond with the first 
word which the stimulus word brings to his mind.) In the 
Garskof and Houston method, subjects were to write down 


their responses. The format was as follows; 
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The method is called a continued association method because 
the subject is to continue writing down all words which come 
to mind by association with the original stimulus word 
within a given time interval. In the task, subjects had one 
minute per page. Pages were presented in random order. 

Using the responses produced, a relatedness index 
was Calculated for each pair of stimulus words. The index is 
based upon the assumption that words which produce common 
responses are related to each other. For each pair of 
stimulus words, the response list (including the stimulus 
word) is numbered according to the following algorithn, 
which will be demonstrated by example. The longer list is 
found. Responses in the longer list are numbered from the 
bottom up as shown below. Starting with the highes+ number 
in the longer list numbering, the shorter list is numbered 


downward. 
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EGG (6) LUNCH (6) 
egg bacon (5) lunch supper (5) 
egg breakfast (4) lunch egg (4) 
egg omelette (3) lunch breakfast (3) 
egg cheese eS) lunch cheese (2) 
egg food (1) lunch 

egg lunch 


Matching responses in the two tests are found. The numerator 
of the RI (relatedness index) is the sum of the cross 
products of the indices of all matches in the lists 
(including the stimulus words). The denominator is one less 
than the sum of all possible cross products if the two lists 
were identical. In the example, there are matches between 
egg-word 6 and lunch-word 4, egg-word 4 and lunch-word 3, 
and egg-word 2 and lunch-word 2. Thus, the numerator of the 
RI is thus (6 x 4) + (4 x 3) + (2 x 2). The denominator is 
SFG 7e-6GYt (5° 3S) Fecewa (7-E TH-F) 5 That As; sthke’ sum ofthe 
squares of the integers from one to the end of the longer 
list, less one. In the example above, the RI = 0.444, 
Proceeding in the above manner, a matrix of 
relatedness indices is calculated for all possible pairs of 
stimulus words. Garskof and Houston reported two experiments 
validating the technigue. In the first experiment, 20 
undergraduates asociated to each member of 24 pairs of 
nouns. The pairs of words had been chosen, eight in each 
group to be “high related” (synonyms), “medium related", and 
"low related". A thesaurus was used to pair the words. They 
came from no particular concept domain. Subjects also rated 


the Similarity of each of the 24 pairs of words. 
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Correlations for individual subjects between relatedness 
indices and similarity ratings ranged from 0.63 to 0.90. In 
the second experiment, uSing the same subjects, Garskof and 
Houston found one day test-retest results on the association 
test ranging from 0.72 to 0.90. While it may be argued that 
the time between tests was too short to discount memory 
effects as a cause of stability of the measure, such data 
still speak to the stability of the word association 
technique. 

Converging results between the word association 
technique and similarity rating techniques offer convergent 
multimethod validity for the methods of assessing cognitive 
structure. Word association testing is based upon the view 
of cognitive structure as a nomological network (cf., 
Torgerson, 1958) of interrelated concepts whose 
interrelationships help form the definitions of the 
concepts. Concept dissimilarity is viewed as the number of 
concepts which must be "passed through" in traversing the 
nomological network from one concept to the other. Responses 
to association tests are "nearest neighbours" to the 
stimulus in the network. Shavelson and Stanton (1975) expand 
on this notion from an information processing perspective. 
If cognitive structure is a valid construct, then a direct 
rating of similarity of concepts from within that structure 
should correlate highly with the association technique, 
which takes as a measure of relatedness the number and 


proximity of "nearest neighbours" in common, as offered as 
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responses to the stimulus concept. 


2-42 Similarity Judgme 
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Similarity judgment tests involve subjects 
responding to all possible pairs of a set of stimulus words. 


The items are usually presented in the following format: 


—_ me re nee re ee ee ee we ee ee ee a ee ee eee ee 


closely related unrelated 
Subjects are asked to put a mark on the line according to 
their judgment of the similarity of the stimulus pair. 
Measures of dissimilarity are taken by measuring from the 
left hand edge of the line in an arbitrary unit of measure. 

In the present study, subjects gave similarity 
ratings to the 105 possible pairs of the 15 concepts under 
examination. The format described was followed. The data 
formed input to the scaling technigues described in section 
2.3. As noted in section 2.41, convergent results between 
association data and similarity judgment data are evidence 
of the construct validity of cognitive structure as defined. 

2-43 Semantic Differential Test 

Semantic Differential technique {Osgood et al, 1957) 
is based upon the following three assumptions: 
(1) The process of description or judgment can be conceived 
as the allocation of a concept to an experiential continuun. 
(2) Many different experiential continua, or ways in which 
meaning vary, are essentially equivalent and hence may be 


represented in a single dimension. 
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(3) A limited number of such continua can be used to define 
a semantic space within which the connotative meanings of 
any concept can be specified. 

In the technique, subjects are asked to respond to 
stimulus words (usually nouns) in terms of seven point 
bipolar scales demarcated by a pair of antonymous 
adjectives. The format is demonstrated by the example below. 

HYPOTHESIS 
SRAED So = 38 Oe 86 6 Fe Fe = FF 
Examples of such pairs of adjectives are: good-bad, weak- 
strong, hot-cold etc. The seven point scale is defined, 
using the good-bad example, as (1) extremely good, (2) quite 
good, (3) slightly good, (4) neutral between good and bad, 
(5) slightly bad, (6) quite bad, and (7) extremely bad. 

Osgood et al have amassed a large body of evidence 
which demonstrates that a substantial proportion of the 
variance of semantic differential judgments could be 
accounted for by three dimensions: evaluative (e.g., good- 
bad), potency (e.9g., strong-weak), and activity (e.g., fast- 
slow). The three will be referred to below as EPA. The 
generality of the EPA dimensions, as reported by Wiggins and 
Fishbein (1969), has since been shown to stand up across 
concept domains, languages, and cultures (Triandis and 
Osgood, 1958; Tanaka and Osgood, 1965; Jakobovits, 1966) 

Wiggins and Fishbein (1969) used the Tucker analysis 
on a set of semantic differential responses to 15 


interpersonal concepts as rated by 97 subjects. Initial 
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analysis of group data revealed the usual EPA three factor 
pattern, which accounted for 73 per cent of group variance. 
Individual analysis, based upon ten “idealized individuals", 
revealed differences in dimensionality, differences in the 
order of emergence of the factors, and differences in the 
factors on which the scales loaded for different 
individuals. For individual responses, the percentage 
variance accounted for by the EPA factors varied greatly. 
For example: in one subject, 33 percent FE, 31 percent P, and 
two A factors, 24 and 10 percent; in another subject, 50 
percent EF, 26 percent P, and no A_ factor; in another 
subject, two E factors, 39 and seven percent, 17 percent P, 
and 32 percent A; and in another subject, 29 percent F, 14 
percent P, and 56 percent A. 

The large body of semantic differential work in the 
literature surveyed has not been related to other techniques 
for uncovering semantic space. Convergence of semantic 
differential data to that produced by the other two 
techniques used in the present study would support the 
convergent validity of the constructs cognitive structure 
and semantic space. It is reasonable to assume that these 
two constructs overlap considerably. Such convergence would 
offer multimethod validity, as semantic differential 
research comes from a radically different research tradition 
and body of literature than either word association testing 


or similarity judgment rating. 
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2.5 Personality Variables 

The underlying assumption of this study is that 
knowledge of a student's cognitive structure may be a useful 
tool in planning for and improving his education. The 
personality variables used in this investigation have been 
chosen because they have been shown in previous research to 
have the potential to be of educational significance. (Bart, 
1971; Messick, 1970) These measures will be used in 
attempting a theoretically based explanation of the nature 
of the subjects as related to their cognitive structure 
results. Some correlates are obvious and traditional. These 
are age, grade, I. 9., sex, and science achievement. These 
will not be discussed further at this point. The personality 
variables of major theoretical interest are developmental 
level, analytic-synthetic ability, and cognitive complexity. 
Developmental level is of theoretical interest for two 
reasons. First, it has been linked to achievement (Lawson 
and Renner, 1974, 1975), and achievement in turn has heen 
linked to differences in cognitive structure (Johnson, 1965; 
Shavelson, 1972). Second, developmental level has been 
linked to the understanding of analogies (Lunzer, 1962), 
which in turn is an important aspect of the understanding of 
the meaning of concepts. Field dependence-independence is of 
theoretical interest in that it also has been linked to 
achievement (Hammond, 1976). The notion of acuteness of 
perception in a visual context carrying over into acuteness 


of perception of meaning has been investigated at length 
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(See Hammond, 1975 and Messick, 1970). Cognitive complexity 
is of theoretical interest in that there is an intuitive 
expectation that complexity of perception of interpersonal 
relationships will transfer to cognitive situations. 

2:51 Developmental Level Test 

Inhelder and Piaget (1958) have defined formal 
operational ability as the ability to consider all the 
possible relationships in a situation, rather than merely 
the actual relationships. That is, a formal thinker can 
think beyond the perceived actualities of a situation and 
consider hypothetical possibilities. Further, he has the 
ability to think in a hypothetico-deductive manner (Lawson, 
1976). According to [Inhelder and Piaget (1958), formal 
thought begins to appear in adolescents at about the age of 
11 or 12, and is fully functional by about the age of 16. 
Research done outside the Genevan school has tended to show 
that Piaget's age estimates are optimistic for more typical 
populations. Such research has been quite thoroughly 
reviewed by Hobbs (1975), and need not be reviewed in detail 
here. However, a few comments must be made about the 
fundamental inadequacies of this line of research, 
particularly as it attempts to uncover the factor structure 
of developmental level. 

AS a generalization, Lawson and Renner (1975) 
estimated that 40 to 75 percent of secondary students have 
failed to reach the level of formal thought. In their 


research, Lawson and Renner have shown the importance of 
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formal thinking ability in the high school science 
curriculum. Using a panel of experts, concepts from the 
biology, chemistry, and physics curricula were divided into 
concrete and formal concepts. A sample of 51 biology, 50 
chemistry, and 33 physics senior high students were 
categorized by developmental level using four individually 
administered tasks. Subjects were graded on a scale from one 
to five on each task, and were then assigned to one of seven 
groups ranging from early concrete to fully formal. These 
groups were, in order: early, transitional, late, and post 
concrete; early, transitional, and late formal. The subjects 
were administered tests of concrete and formal subject 
matter concepts. The results were interpreted by Lawson and 
Renner to show that concrete students could understand only 
concrete concepts, while formal students could understand 
both formal and concrete concepts. In a table of percentage 
correctly answered concrete questions VS. level of 
development, there is a gradual increase from about 30 
percent for transitional concrete to 80 percent for late 
formal. In a similar table for the formal questions, 
virtually no subjects below post-concrete had any questions 
right, but beyond this point, there was a regular increase 
from 15 percent for post-concrete to 45 percent for late 
formal. The sample is, however, too small for the formation 
of any firm generalizations. Because of the connection 
between achievement and formal thinking ability, it is 


important to look for a connection between formal thinking 
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ability and cognitive structure. If cognitive structure is 
to bear any relationship to subject matter achievement, it 
appears reasonable to expect differences in cognitive 
structure between concrete and formal thinkers. 

It is not completely clear from recent research 
whether formal ability is unifactor or not. Lawson and 
Renner (1975) maintained that the conservation of volume 
task (Karplus and Lavetelli, 1969), the chemicals task and 
the balance task (Inhelder and Piaget, 1958) are essentially 
measuring the same ability. Lawson and Renner used stepwise 
multiple regression analysis to test the improvement on 
prediction of achievement by one Piagetian task if further 
tasks are included as predictors. For their six achievement 
tests (concrete and formal biology, physics, and chemistry) 
the addition of a second predictor did not significantly 
improve the squared multiple R. In three of the six cases, 
the best predictor was the conservation of volume task, and 
in the other three, the chemicals task. Only one of the six 
squared R's is reported, at 0.41. Principal components 
analysis of their data yielded a first factor which 
accounted for 62.2 percent of the total variance. Thus, they 
suggested that subjects can be satisfactorily characterized 
by only one task. 

Bart gave the shadows task, the balance task, the 
pendulum task, and the conservation task (Inhelder and 
Piaget, 1958) to a sample of 30 above average subjects from 


each of the age groups 13, 16, and 19 years. Not enough 
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detail is provided to calculate the percent trace, but 
maximum likelihood factor analysis produced a first factor 
on which the four tests loaded with coefficients ranging 
from 0.62 to 0.88. Bart interpreted these data as indicating 
the unifactor structure of the four tasks. Bart's subjects 
also wrote three subject matter tests of formal operational 
ability. When all seven tests and tasks were factor analysed 
together, all seven loaded on the first factor with loadings 
between 0.54 and 0.84. The tests loaded on the second factor 
(28 percent as large as the first factor) with coefficients 
of 0.24, 0.31, and 0.51. The tasks loaded on the second 
factor with coefficients fron 6-0 .42— tor ]On92s / Bart 
interpreted this to indicate that there was a ‘large formal 
operational factor and a smaller test writing factor in 
operation. He did not mention the possibility that the first 
factor was a general intelligence factor. 

Lawson and Nordland (1976) administered ten tasks to 
96 grade seven subjects. In a principal components analysis, 
their first two factors, identified by them as concrete and 
early formal factors, accounted for 55 percent of the total 
variance. The second factor was called "early" formal 
because only three percent of their sample had reached the 
early formal level of development, as judged by their 
results. All of these studies used individually administered 
tasks, scored on a scale of four or five points. 

The need for efficiency of testing in large scale 


studies has led to the development of paper and pencil group 
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tests in place of the individually administered tests of 
formal operations. Raven and Polanski (1974) using Raven's 
(1973) paper and pencil Tests of Logical Operations (RTLO), 
a mixture of concrete and formal tasks, required five 
factors to account (only partially satisfactorily) for the 
ability measures. These factors accounted for a total of 40 
percent of the variance in the total test battery, which 
included eight other tests as well as the seven subtests of 
the RTLO. One possible reason for the disparity between the 
individually administered test results and those of Raven 
and Polanski may be in the scoring system used. The former 
usually score subjects on a restricted scoring scale (e.g., 
1 to 4 for early and late concrete, and early and Ilate 
formal). The RTLO is not restricted to such a narrow range 
of scores. The seven subtests contain from eight to sixteen 
items each. Although scoring procedures are not provided in 
sufficient detail in the report, it is possible to picture 
an implicit rule of thumb operating of the nature: "If a 
subject is not clearly concrete (2) or not clearly formal 
(4), give him a ‘'3'." Such a procedure, a function of the 
scoring system used and not of the nature of the test (i.e., 
group or individual) could lead to artificially high 
correlations among tests. This might account for the 
unifactor structure found in the above research. The RTLO, 
on the other hand, with more than one item per conceptually 
distinct ability, may be avoiding the potential problem 


caused by a restricted scoring system. It is difficult to 
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dismiss the very high loadings which some of the subtests of 
the RTLO have on some of the factors found in Raven and 
Polanski'ts study. Without better empirical evidence, the 
factor structure of developmental level can be no more than 
speculation. 

The point of the speculation, however, is to comment 
on the state of the art in the measurement of developmental 
level. On the basis of the present evidence, the factor 
structure of formal operations remains an open question. The 
intuitive appeal of Lawson and Renner's subject 
characterizations, however, give some measure of support to 
the use of restricted scoring systems for the purpose of 
classifying subjects. It seems reasonable to assume that a 
weighted composite of a representative sample of formal 
operational tasks can produce sensible subject 
characterizations, despite the lack of consensus on the 
factor structure of formal operations. Some of the problem 
concerning the structure of formal operations may rest with 
the application of metric factor analytic techniques to the 
ordinal data produced by the restricted scoring systems. 
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Cognitive style variables have attracted attention 
of researchers for some years (Messick, 1970). In general, 
they are an attempt to account for components of ability not 
accounted for by intelligence. Analytic-synthetic ability is 
the general name for a large number of variables, including 


cognitive control (Gardner, 1973; Gardner and tLohrenz, 
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1969), scanning (Holzmann, 1966), flexibility of closure 
(Field and Cropley, 1969), levelling-sharpening (Lohrenz and 
Gardner, 1973), and, most important and longest-lived, field 
independence-dependence (Witkin, 1950). Hammond (1976) has 
provided a substantial and detailed review of the area which 
need not be repeated. The field, according to Klein et al 
(1967) suffers from an excess of conceptual and empirical 
overlap coupled with many competing definitions. 

In an effort to bring empirical clarity to the area, 
Messick and French (1975) administered a battery of 34 tests 
of speed and flexibility of visual, semantic, and verbal 
closure to 541 naval cadets. Principal axis factor analysis 
produced a fourteen factor solution, accounting for 46 
percent of the variance. The first four factors had 
eigenvalues greater than one and accounted for 33 percent of 
the variance. These factors were labelled ideational 
fluency, flexibility of perceptual closure, flexibility of 
verbal closure, and flexibility of semantic closure. 
Intercorrelations among the first four factors ranged from 
0.10 to 0.25. Nine of the thirty-four tests loaded 
Significantly on the second factor, which is the one of most 
interest in the present context. None of the loadings was 
higher than 0.62. The Hidden Figures Test used in the 
present study was not part of Messick and French's battery. 
A factor solution such as that reported by Messick and 
French is testimony to a lack of adequate isolation of the 


dimensions of cognitive and perceptual closure by available 
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tests. Despite the difficulties involved in finding 
relationships, educators have clung to an intuitively held 
belief that the concept of field independence must be 
educationally important. This belief is not without 
empirical support. Satterly (1976) suggested that the reason 
for this belief lies in assuming a link between the ability 
to perceive figure apart from ground with the ability to 
"experience information as discrete from the organized 
context of which it is a part," and the lack of this ability 
as a parallel lack of cognitive ability. Vernon (1972) 
pointed out that interest in this dimension fror an 
educational perspective is justified only if analytic- 
synthetic ability provides a dimension distinct from 
intelligence. 

Based upon his review, Hammond (1976) concluded that 
there is enough evidence that the field independence 
cognitive style variable is distinct from intelligence to 
warrant further research. Using six classes of grade 12 
physics students, Hammond found field independence, as 
measured by the Hidden Figures Test (French et al , 1965), 
which is a group version of the Embedded Figures Test 
(Witkin, 1954) unrelated to verbal and mathematical ability, 
but related to physics achievement. 

Using a sample of 201 boys age 10 and 11, Satterly 
(1976) found substantial overlap between intelligence and 
field independence (r=0.41; p<.01), but significant residual 


correlation between field independence and both mathematics 
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achievement and haptic perception after intelligence was 
partialled out. 

Because of the Significant but unimpressive 
correlations (generally 0.30 to 0.60) among various measures 
of analytic-synthetic dimension, Wachtel (1972) recommended 
the use of several measures and a shared variance technique 
to assess field independence. Due to time and manpower 
limitations, this suggestion has not been followed in the 
present study. Field inde pendence- dependence will be 
assessed, as in the Hammond study, by the Hidden Figures 
Test. It should be noted that the Hidden Figures Test 
correlates with the Embedded Figures Test, which may be 
taken as at least having attained the dignity of age, at 
r= 0.62 (Davis, 1972). This is one of the largest 
correlations to be found in this psychometrically and 
conceptually unsatisfactory area of research. 

Based upon the generality of perceptual skills 
between both the visual and cognitive domains, it is 
hypothesized that the meanings of concepts perceived by 
those scoring high and low on the Hidden Figures Test will 
differ from each other. It may be further hypothesized that 
these differences in perceptual patterns will be 
interpretable in terms of the meanings of the concepts 


rated. 
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-53 Cognitive Complexity Test 

Cognitive complexity is a cognitive style variable 
Which deals with the degree of complexity with which a 
subject views phenomena, that is, the number of bases or 
dimensions that a subject uses to make decisions. Of the 
three main personality variables used in this study 
cognitive complexity is intuitively the one most likely to 
be closely associated with cognitive structure. Subjects 
with a many dimensioned cognitive structure would be 
considered cognitively complex, while those with a cognitive 
structure of few dimensons would be considered cognitively 
Simple. Unfortunately, there seems to be little agreement in 
the literature as to how to measure cognitive complexity. 

As Kass has pointed out (1969) in a review of 
cognitive preference and cognitive complexity, cognitive 
complexity has arisen from a tradition of personality 
measurement rather than cognitive measurement. The 
complexity of judgments in social interaction rather than 
subject matter content has been studied. However, it seems 
reasonable to expect a transfer of complexity, viewed as a 
multifaceted judgmental ability, between the two areas. This 
expectation of transfer is examined in this study. 

Klein et al (1967) has pointed out semantic and 
conceptual overlap of the terms associated with the field. 
They point out commonality among conceptual systems (Harvey, 
Hunt, and Schroder, 1961), psychological differentiation 


(Witkin, 1962), conceptual differentiation (Gardner and 


: 
eitg ice siyss oisdatiedl wat Denitetakae . 
@ doidw (618 ata YO. eaenee aut Pee een doidy _ 
so 0 6t9ee8 «(Fo  redeun ais yeh fad? ~fonaoned@a Swoiv fostdua | 
oi) 6 epiwintaeh of68 on aaeu foefduc 6 dads anofenomib 
yin ta ait? ai Seeo aekdstasvy yttiseosiseq aise: sotdt 
a< vfesél took ono ade ybovetivu’ni 2t wiixelqmoo ovitingoo 


joudédwe ce tmdowrte avktiopos dtiv Shedtetoosss y lezolo od 


-Tyew omton7*te avitiapan beaotenenth Yaum & dviw 
ovltinnen o ftiw enod? of btw (Xolgno> yloviticponm hetshtanes 

citiacoy fertebiedoo od bhuow snoenonth #91 to stoto0728 

propenron efitht ad oF amece atedt ,yletsautioiall elqaia 

4+ ixefameon avitiapas stuzeon of vod of se Stes 7EeRE oft 

poives 6 ai (PORT) «tuo hosatoqg wth anew e8A- 

tisonn ,wdaxetames evitinpon bos canete ler avitinapoo 
vetiveoore: Yo sottihett 6 sort aerits asd ytineiqaes 


uit ‘rome ye een eviaiapoo newt Tot tar tne sTuZ698 


, 


i 


neti tedtes dodtoatata®, [etsoe ai stnomphyt to ytixelagon 
hook 21 ,2evewot sferhyte agad and +totnon tettsa doetdie 
, on howetv ,yiixsefamon Yo astenstt 5 toeqxe of eldsronso1 
Edy Jmeate ows o¢7 deowted yytifids Issnoapbot botesostitiva 
svhute etd? ab honkwexo ai tetanstt to noktetoeqges 
hav Etsemen tio toaatog est (TaeTY Le te meelh- 
Lpiot® ede dtiv: botetuowas me ait to abieigiess Leo H : 103 
youste sh ‘Gaetiye Leuteannes |  YPELENOREOD 70 sakog yd 
ne bted tors’ ath Isoipofodoyag y (TACT yz9homine Bas st 
hus reaband) sokteitnat@A4b feutasono> (SAET 


68 


Schoen, 1962), and cognitive complexity (Bieri et al, 1966). 
in their review, they conclude that there is 1ittle 
correlation: among measures of cognitive complexity. 

In reaction to this state of affairs, Smith and 
Leach (1972) have proposed a new measure of cognitive 
complexity based upon the Repertory Grid System (Kelly, 
1955). Kelly‘s system is also the basis for the measure of 
Bieri et al (1966). Smith and Leach, however, criticize 
Bieri's measure as a measure of differentiation rather than 
complexity. An example of the Bieri technique involves 
having subjects generate a list of several significant 
people in their lives, anda list of constructs on which to 
judge these people. Numerical judgments are placed in a 
people by constructs matrix. The test is scored by counting 
the number of times a person is rated the same on a pair of 
constructs. The -total matching score for all pairs of 
constructs defines Bieri's measure of cognitive complexity. 
A subject with a high matching score tends to use all his 
constructs in a similar manner, and so is deemed cognitively 
Simple. Similarly, subjects with low matching scores are 
deemed cognitively complex. Quoting evidence based upon 
research with schizophrenics, Smith and Leach suggested that 
the Bieri measure confuses "complexity" with "confusion." To 
remedy this, Smith and Leach began with a large (in this 
exagple, 13 by 13) matrix of people {to be rated) by 
concepts (on which to rate them). With subjects supplying 


both people to be rated and concepts, ratings were solicited 
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on @mctscaie sof “eh2 4at=22A 913 «13 ‘mateix of correlations 
between constructs was produced from this data. Thus, a 
correlation gave a measure of similarity of constructs as 
used in the rating task. At this point in the analysis, 
Smith and Leach claimed that they had something analogous to 
Bieri's measure, that is, a measure of differentiation. They 
hypothesized that "the fine details of the construct system 
will be more important for a complex subject than for a 
simple subject, so that impoverishing the structure will 
have a more dramatic effect on the relationships between the 
people for a more complex subject". To impoverish the 
structure, all significant correlations were collapsed using 
Se | Gao akohnson’?s (1967) hierarchical clustering. In one 
experiment (N = 27), this measure was found to correlate 
0.10 with Bieri's measure of complexity. However, it was 
found to be significantly related, by the Mann-Whitney JU 
Test (Sigel, 1956) to Harvey's (1967) TIB Test, a 
projective, paragraph completion test which arose from the 
early Harvey, Hunt and Schroeder (1961) work. Bieri’s 
measure did not correlate with the Harvey test. However, in 
a second experiment, (N = 42) the Smith and Leach measure 
was found not to generalize across content domains, while 
the Bieri measure did. The correlation between judging 
people and other topics of interest (e.g., novels) was, by 
the Bieri measure, 0.35. This finding has significance for 
the present study in that cognitive complexity, as mentioned 


above, must generalize from interpersonal judgments to 
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subject matter judgments to be educationally relevant. Ina 
third experiment, (N = 15) Smith and Leach found one-week 
test-retest reliabilities of 0.76 for their measure and 0.46 


for the Bieri measure. 


2.6 Summary 

This chapter has reviewed the literature concerned 
with concept relationship research in science education. 
Also, the scaling techniques used in the present study were 
reviewed, together with literature on the six tests used in 
the present study. 

In summary, the present study has gathered concept 
relationship judgments from subjects by three different 
methods. From these data, cognitive structures, defined as 
geometric representations of relationships among concepts, 
were produced by two multidimensional scaling techniques 
which are based upon different assumptions. Differences 
among subjects in their perceptions of the concepts studied 
were related to personality data on the subjects which was 
gathered by three independent tests. The next chapter 
presents details of the methods employed in carrying out the 


study. 
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CHAPTER 3 


PROCEDURE 


3.217 Introduction 

This chapter outlines the actual procedures used in 
the carrying out of the study. The development of the 
instruments used and the pilot work involved are discussed 
first. The next section outlines the methods used to 
estimate the reliability of the instruments, along with 
reliability estimates found in other studies using the same 
or closely related instruments. The section which follows 
discusses the design of the study and the sample used. 
Section 3.5 presents the details of the scoring procedures, 
and gives justification for their use. Section 3.6 does the 


same for the analytic procedures. 


3.2 Instrument Development 

221 Constrained Association Test 

The set of concepts chosen for this study are 
Syntactical rather than substantive in nature. Because of 
this, they should be free of some of the subject matter 
constraints found by Kass (1971) when content oriented 


concepts were used. Although the concepts themselves are 


interesting as a subject of study, they serve primarily as a 


71 


a7 
mn . % 
APs ie & 47) 
f) METSRHD ee. ho 
a = 
SA0G9 Dar) 
aoitoubottaT Fee. 
wi been seruhesoiq Peutos edt sent ltuo as4anqdo afd? ie 


oft Yo Saangoleveh ed Vite att Yo tno paiysts> off 
beeeurn th ot: ‘hevfovat jtow tolin of? Sas bony etnsanztent 
nd oben ahoddon ett feakitve notioen #xe0 adt dark? 
f4te onofe  etesmutgeadk edt %o ytilideting edt 9 jealtes 
acee e¢* paren aokbute redto mi bar eotemtses ytilideiler 
wyaTin? dotdy oodtoen ad? cetaomorteni betsies ylagoka go 
sen ofeaes edt Das Gap ae eds to apieeh od? aseavoalb 
oatitesota pottodal ed? Fo .efietsh of? aineeew 2#.€ nolioe2 
a#* ouch Jef aek49@8 am tfed4 70% coiteoi@iaent aevip. hes 
zaiuho nog aityl sah a3 10? 9m628 

oe 

tnangolLeved tromuttant Sot 

inet uobtpkooess hanisategod bSab) 

oes bute eid’ 107 naRddd ntagonoo to fee odf | _ 
to manEjet, oTuten of ovitastadie nbdt tedses Leoistostaya 


468 toerpdue ott I ome to o9s2 od biuoda god? abet 
Pa fanart “tao400> aeadw ceTety homes - =r ae a 


ws eovfeemedt etqennoo al? 
r se YLivbmbzn ove vod? af 


42 


vehicle for the investigation of the methodology proposed in 
this investigation. 

The study was begun by generating a list of concepts 
associated with the scientific method. Some, such as 
"evidence" and "conclusion" were expected to be in common 
use in science classrooms. The initial hypothesis in mind at 
the time of concept selection was an hypothesis of two 
dimensions of variation of the concepts, a time dimension, 
and a certainty dimension. 

The feasibility of the study was established by 
three pilot studies, two informal and one formal. The first 
involved a group of nine graduate students and faculty 
members in a science education graduate seminar group. This 
group performed a modified version of the Kelly test (Smith 
and Leach, 1972). Fach member of the group was given seven 
triplets of concepts chosen at random from the set of 15 
used for the study. For each triplet, the subject was asked 
to give a way in which one of the three differed from the 
other two. An analysis by the investigator of the dimensions 
of difference indicated by the subjects found the two main 
clusters as hypothesized. A third, considerably smaller 
cluster, might have been labelled as theoretical vs. 
practical. These three clusters accounted for about 90 
percent of the responses to the test. 

A second pilot study involved a group of 19 physical 
science student teachers. All members of the group had at 


least one degree in the physical sciences. Subjects were 
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asked to respond by word association to the 15 concepts used 
in the study. Concepts were presented ina different random 
order to each subject. Responses were in written form. 
Concepts appeared one to a page, with a column of blank 
spaces for responses under each. Subjects responded to each 
concept for one minute. The subjects were constrained to 
respond with "scientific words". This constraint was later 
modified to “think of the scientific meaning of the word and 
respond with the first word which comes to mind". This study 
focused on fluency of response, and also produced pilot data 
for use in testing and debugging computer programs used in 
the analysis. As a result of this pilot study, it was 
established by chi-square test that there was no significant 
difference in number of responses related to either stimulus 
concept or order of presentation. There were, as would be 
expected, significant differences among subjects. Kruskal 
analysis of the group average relatedness indices produced 
by this group revealed an easily interpreted spatial 
representation in either two or three dimensions. It was 
found, however, that the hypothesized dimensions of time and 
certainty were confounded. In retrospect, this seems 
entirely reasonable. As a scientific enterprise proceeds in 
time from, for example, "puzzle" to "law" or "conclusion", 
the degree of certainty also progresses. 

The third pilot testing involved three grade 9 
science and three grade 12 biology classes. Fach class in 


each grade completed two of the six tests which were 
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administered to the study sample. This testing was useful in 
establishing timing, clarifying directions, practising 
protocols, and providing, in conjunction with a return visit 
to the grade 9 classes one month later, some test-retest 
data. These data will be described below. 

The final version of the constrained association 
test was presented using the modified constraint just 
described. That is, subjects were asked to "think of the 
scientific meaning" of the concept rather than to "respond 
with scientific words". There were fifteen different random 
orders of pages, each beginning with a different concept. 
The repeated pages were the first page and second last page 
of the booklet. The first page was used in the primary 
analysis. Appendix A presents the directions and a sample 
page of the text. With one minute per concept, and one 
concept repeated as a check on consistency, the test took 16 
minutes. With time for distribution, practice, and 
collection, administration time was about 25 minutes. The 
final test form was photo-reduced by half from the original 
typewritten copy on 16 pages of 8 1/2 by 5 1/2 inch paper. 
There was no evidence of inability to read the reduced type 
on the part of the subjects. 

22 Similarity Judgment Test 

There are 105 possible pairs of the 15 concepts of 
interest. In pilot work with the student teacher group, 
concept pairs were presented seven to the page, with two 


pages for practice, for a total of 17 pages. The practice 


ar : oa 


ot! freeer eww partos*, se si yhure 
Pdtr EAT ganed sien FPR, wabyl esots ~ wes fad e | 
eppivy avoten « 4s 2k ua ni pars tvor at ener 
tuatan-*298% Spo ite iftem ano anne it e baa ets of 
wolsd Redtasesh od [fiw ste enodt .6ae6 : 
sobtnbnobas hep tertanen 68S fo noieiov feat? oT : 
tout ‘tetertenon Sobiibog ef4 potas fhetases7q B69" tear, 
44 %a Sng4t” of fotee otaw SBihopdve 2k deat fodianesh 
hnouset™ of met? ta€4962 tq99H0n edt to ‘aimee ott itnetoe 
ehyet too78T Th wettsy ee aqad? ."ehtow sititaetsea dsiw 
‘eer ‘qevet?ibh #6 #¢fw vatrantpsd dose ,2spag Io si1abI08 


ope S2eP Deopea hap epne deaTr? 6d2 st9w soepag hetseget od? 


} Ton wt ar horn aay gpac #22f? of? te ftood ait to 

alawes « bee snoktoet kh ad4 etnesatq 4 xi basagh sateylsoos 

ait .  ,tasonen eq atonie sno dtiW ~txot add Yo epeq 

i} dnot +264 9d¢ (¢onatefadow aco Aneds 5 26 betheqad tq90no5 : 

hae ,eottreta ob Pode ariesh 3702, oni? ddi¥ -aetonia 

oi? .eaivire @S) tuede eBy-emk? no iterte tatebe .tok tool foo ; 
: 


‘haippen ot* wort ted yd haowhst-ofodq saw wt08 too? LIenit 
eg dook St @ ydoS\t be eepaq of no yqon satdiweqyt 
vt Pepohas of¢ boot of gotlident Io ovasbive om sew “oTeet 
spiostdus oft Jo d36q- edd 90 
| team tuempbul ytiaelioge S8a8 eS 
te aAtgendod ?@f att Yo artuq afdtesog ?OT otf sret? 
eto TH sedoget tiobota oft S ohrow tolig “ans 0 
owt dtiw  yereq oft of novo f8 ; 


anit+oerg ott .eepea TT 30 f 


pages came first and were the same as the second and fourth 
last pages. All subjects had the same random order. Scaling 
of group average results showed a good comparison with the 
word association data results. In the final study, a repeat 
for consistency was incorporated. Concepts were presented in 
reduced type on half pages, using both sides of the page. 
There were ten pairs per side, seven pages, for a total of 
140 judgments. Two random orders were used, one the exact 
reverse of the other. The first ten pairs were for practice, 
and were not scored. The last five were repeats introduced 
to fill an exact number of pages, and were ignored. One of 
the seven pages was a repeat, placed either immediately 
after the practice page or last in order to maximize its 
separation from its duplicate. The practice examples were 
repeats from near the end of the test, and the fillers were 
repeats from near the beginning of the test. Because some 
subjects repeated the page with the practice examples, and 
some the page with the fillers, there were subjects who 
repeated either 10, 15, or 20 pairs. In the pilot work, a 
measuring device scoring the ratings on a scale of 25 points 
was used. Since it seemed to offer no improvement over a 
scale of nine points, the scale with larger units was used. 
Such a scale division size is in Keeping with Miller's 
(1956) recommendation. Appendix A presents the directions 


and a sample page of the test. 
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emantic Differential Test 


it 


The pilot version of the semantic differential 
instrument presented 20 scales on which the subjects rated 
15 concepts. These were chosen as follows: four from each of 
the standard dimensions from a list of adjectival pairs and 
the dimensions on which they load (Osgood and Suci, 1955), 
and eight from the descriptions produced by the graduate 
student and faculty pilot study group referred to above. The 
intention was to cover any dimensions expected from the EPA 
factors, and also to cover any dimensions unique to the 
concepts chosen. Pilot work showed that subjects would not 
be able to finish such a lengthy test in the time available, 
so the test was shortened to only twelve adjectival pairs. 
These were chosen to give three for each of the EPA 
dimensions, and three unigne +o the study. According to 
Osgood and Suci'ts (1955) results, the nine scales chosen all 
load highly on one factor only, three on each of the EPA 
factors. According to Maguire's recommendations (1973), all 
subjects were presented with the adjectival pairs in the 
same random order. The concepts were also presented in the 
same random order for all subjects. Since both sides of the 
page were used, it was necessary to introduce a sixteenth 
concept, "generalization", which was ignored in analysis. 
Subjects responded to both sides of nine half pages, 
allowing one repeat for a consistency check. The repeat was 
placed first or last in order to maximize its distance from 


its duplicate. That is, it was first if one of the last 
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pages was repeated, and last if one of the first pages was 
repeated. Examples of the instructions and a sample page can 


be found in Appendix A. 


Despite the "blunt instrument" nature of written 
Piagetian tasks using restricted scoring systems, they are a 
necessity in this investigation due to time constraints and 
sample size. Since no conclusions are to be drawn regarding 
the factorial structure of formal thought, this does not 
impose any great limitation upon the results. 

Despite Lawson and Renner's (1975) recommendation to 
use only one task to characterize subjects, it was decided 
to use a composite score based upon four tasks. If formal 
thought is, in fact, multifactor, then a composite score is 
probably more appropriate. This appears reasonable because, 
in this study, one formal ability can not be said to be 
inherently more important, or relevant, than another. 

The first test item (see Appendix A) is the 
conservation of volume task used by Karplus (Karplus) and 
Lavetelli, 1969). Departures from the administration method 
of Karplus were necessary in order to allow administration 
of the developmental test and one of the concept 
relationship tests within 50 minutes, the time available for 
testing in one junior high period. A modification was added 
because of the group situation and time constraints. 
According to Karplus, after subjects make and explain their 


predictions, they are shown the results of the demonstration 
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and are allowed, if they were wrong, to correct themselves 
by explaining the discrepancy. Despite being told the test 
did not count, the tendency to change responses seemed to 
eccur in a significant number of subjects in pilot work. 
Thus, subjects in the main study were not shown the results 
of the demonstration until their response sheets had been 
collected. There was no opportunity for a second 
explanation. This departure from Karplus' method may cast 
doubt on data based upon this one item only, but seemed 
necessary in view of the circumstances. 

The second item is a combinatorial task taken 
directly from Hobbs (1975). It is a logical equivalent of 
the Inhelder and Piaget (1958) chemicals task. Item 3, a 
controlling variables task, is also from Hobbs, although he 
used it for subject training rather than testing. Iten 4, 
another variety cf controlling variables task, uses a 
diagram from Hobbs, but the questions asked are different. A 
version of Karplus’ "Mr. Tall and Mr. Short™ task {Karplus 
and Lavetelli, 1969) was included in pilot work, but dropped 
because of the time required for its administrastion. A 
composite score, based upon the four items in the final 
version of the test was used in the main study. Details of 
scoring are presented below. The four items were selected to 
represent two of the principal types of reasoning found 
indicative of formal reasoning (Inhelder and Piaget, 1958), 
that is, a conservation of volume task and a combining 


variables task. These tasks are also the tasks found to be 
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the best single predictors of achievement by Lawson and 
Renner (1975). 

The conservation of volume task was administered 
farst; with sall. subjects watching the demonstration 
together. After the first part of the demonstration, 
subjects worked through the test booklet at their own pace. 
Tasks were presented one to a page. There was no time limit, 
but all subjects finished within twenty-five minutes. After 
the booklets had been collected, the demonstration was 
completed. The text of the test appears in Appendix A. 

3.25 Field Dependence~Independence Test 

AS was mentioned above, it was decided to use the 
Hidden Figures Test (French et al, 1965) as a measure of 
this variable. The Hidden Pigures Test, commercially 
available from Educational Testing Service, is a timed test 
of the ability to find simple geometric patterns embedded in 
more complex patterns. The test is in two ten minute halves, 
with sixteen items per half. The format is five-alternative 
multiple choice. The choices, which remain the same 
throughout the test, are printed and labelled A to F at the 
top of each page. Each half of the test occupies two pages, 
which can be viewed at the same time. A correction for 
guessing is applied. With time for distribution, practice, 
and collection, administration time is about 25 to 30 
minutes. Because of the commercial availability of the test, 


is does not appear in the Appendix. 
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3-26 Cognitive Complexity Test 

Seaman and Koenig (1974) produced a system by which 
they compared several measures (that is, scoring systems) of 
cognitive complexity using the same response data. The 
cognitive complexity measure used in this investigation is 
based directly upon one of their scoring systems, and very 
Similar to that of Bieri. Subjects are presented with 
general descriptions of Six significant individuals. 
Examples of such individuals are "person of same sex you 
feel uncomfortable with", "your favourite teacher from this 
year or last", etc. The test presents a series of six 
criteria (out going-shy, decisive-indecisive, friendly- 
unfriendly, cheerful-illhumoured, dominant-submissive, and 
considerate-inconsiderate) with which the subject is to rate 
the six individuals on a scale of one to five. A score of 
cognitive simplicity (the inverse of complexity) is found by 
counting the ties in the scores of each individual on all 
possible pairs of continua. This amounts to a measure of the 
“halo effect" in the subject's rating of the individual. The 
test was administered on one page of paper, with directions 
on one side and the matrix on the other. Subjects were given 
an example and allowed to practise the task. The privacy of 
the people they were rating was carefully preserved by the 
directions. Explanation time for this test was about ten 
minutes, with time for completing it varying from ten to 
twenty minutes. In every junior high class, the investigator 


was asked the meaning of the terms "dominant" and 
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"decisive". These were defined, for the whole class, as 
"bossy" and "able to make decisions", respectively. The text 


of the test is presented in Appendix A. 


Sections 3.24 to 3.26 describe the personality 
variables of major theoretical interest in the 
investigation. The following section discusses other 


variables which were generated from the data during the 


course of the study. They can be divided into two 
categories, those dealing with fluency on the word 
association test, and those dealing with personal 


consistency on the three relationship tests. A good deal of 
variation was observed in the ability of subjects to cope 
with the constrained association task. Because of this, the 
following variables were generated as being potentially 
useful in the analysis: total number of responses produced, 
total number of responses consisting of original keywords, 
percentage of responses using the original keywords, average 
relatedness index generated by the subject, personal 
consistency scores (to be described in section 3.3) for each 
subject on each of the three concept relatedness measures, 
and overall personal reliability (section 3.3). The first, 
second, and fourth of these have been investigated, as 
described in section 2.2, by Johnson (1964, 1965, 1967) and 


Shavelson (1972, 1973, 1974). 
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3.3 Instrument Reliability 

3.31 Constrained Association Test 

As mentioned above, Preece (1976a) reported a test- 
retest reliability over one month of 0.86 for a sample of 
eighteen year old physics students (number in sample not 
reported). Since the stability of a group average depends 
upon the sample size, such a figure is difficult to 
interpret. There is little available in the literature 
regarding the reliability of individual measures of word 
association tests. 

The reliability of the data has been estimated in 
several ways, all involving test-retest stability. Test- 
retest data over one month are available on a sample of 
nineteen grade nine subjects used in the pilot work. This 
reliability may be reported in several ways: first, and 
yielding the lowest "reliability" would be an average over 
all subjects of the correlations between relatedness indices 
produced by each subject on the two occasions; second, a 
Similar calculation for group average data; third, 
correlations between the loadings of the concepts as a 
result of Kruskal analysis of the group average data for 
each occasion; and fourth, a goodness of fit measure using 
the nonmetric fit test described above for group average 
data over the two occasions. 

In the main study each subject responded twice to 
one of the concepts. A relatedness index calculated between 


the two sets of responses (adjusted to compensate for the 
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fact that the first words in the two lists are always 
identical) gives a measure of the subject's personal 
consistency on the one testing occasion. Such a measure is 
crude at best, and certainly is itself subject to 
indeterminate measuring error. In retrospect, it would have 
been worthwhile lengthening the test by two minutes and 
getting duplicate responses for three rather than one word. 
However, the calculation which was done may have some 
meaning as a measure of consistency. 
arity Judgment Test 

The literature surveyed does not report data on the 
reliability of similarity judgments. Johnson (1967) reported 
correlations between word association data and similarity 
judgment data in the order of 0.70. A possible way of 
estimating reliability would be to calculate the goodness of 
fit between individual matrices and the group average. Such 
a procedure is rejected because, in addition to equating 
conformity with reliability, it would be costly in terms of 
computer time. The method, limited though it is, chosen for 
this study is to report the Pearson product-moment 
correlation between the ratings for each individual over the 
10, 15, or 20 pairs which he rated twice. Correlations were 
adjusted by the Spearman-Brown formula to give a consistency 
measure based upon 20 repetitions. While it would be 
possible to calculate the stepped-up correlation as if the 
repetition were over all 105 pairs of concepts, this was not 


considered warranted. 


eed oils we ehaow tend ag 
haunorsad >! (neh cae eat ahddusect & sow bp . op ot te 
i seam © ONT shai paektea? ono ott abcde } 
64 tsoPeue Fleets nie yinis-ties fers aaa 46 obi 


ave bition th .toseROTT RINE ste parteerom ‘eteninsetebad: 


gyawte Oa attri 


hoe aebprte 681 ya det ett pata tread sfidvdtiow peed 
iptow) ear hd? tote oa eords 402 aeenoqrad eteobiqub parttep. 
vaie Oved fo enoh” Bee ioidw coitelootes “edt . TSvewoll 


<Yornaturenos to ATU2bOn 8 es poinsen 


vt , 
toot gapmphub ytimelimge Shel ‘ 
ad* eo pofeh *1oqe? ton a90h heyevi1.e erpinvetil ead? 


(Coot) eoandob. ataompbhut ytinsl ite to ytiLidsifes 
w+pvetTinic hoe #eBB anitstsores fhrow noewted anottsierzo0o 

vey ocldiaeon Ao .PNa0° Yo asbro eit ak steb taempbnt 
‘ ssenkoop od¢ ¢¢etaoiss at od fhisow ytilidetior pai tsatteo 
tay’ .angreve ctotn off fap 2opta7sa teubivibat seowted 2i1 
sai*bies (of nod hthbe Ue soanesnod hotpetet af stubs omg & 

wre &4 YPoson oad bipow of WWiilidnt Los ase leicht 
ot garody jer th dpgodd” be tiest vbodten od? soett astaqao9 7 
soyune-sorhotg sooeneed| Bdt = tr0ge1 of sf ~ yhuge ebdt : 
ait qevo ferhitvihat dose 202 epaids1 oft neovted nok? slor209 © 
aco Soot+efet te) .solwt bets? ed doite aatieg Os 20 EOE 
voayetetudon 6 evin ot »ipm202- vert -nontanas ede nditeanatha) 
ef Alnow *f ofbAv 2n0bskt6Q9% oS noge 
edd Tb nee cw begged? aia 
ton Saw meds .etoennon Io 27k @or Ife aan — 


re Gn ee 4 


84 


DiVesta and Dick (1966) and Norman (1959) have 
investigated the stability of semantic differential scales. 
DiVesta and Dick measured four-week test-retest stability 
using a total score from the two scales which loaded highest 
on each of five factors. The sample of 500 children, grades 
2 to 7, rated 100 concepts on 27 scales. Stability estimates 
averaged over the five factors ranged from 0.30 for grade 2 
to 0.55 for grade 7. Stabilities for the EPA factors in 
Stage? were: (6,0. 787°R,°9.55: 2, ° 9.58. 

Norman (1959) proposed the use of a stability 
estimate which measures the actual size of shifts in the 
rating of each concept on each scale divided by the maximun 
shift possible. Using 30 adults rating 20 concepts on 20 
scales his measure produced less than 30% shift in all 
cases. Pearson product moment correlations on the same data 
gave correlations of the order of 0.66. Since the Pearson r 
is more easily interpreted, there seems to be no advantage 
to using Norman's other stability measure. When Norman moved 
from stability of scales to stability of factors, stability 
of the EPA factors (using 8,3, and 3 scales) were in the 
range of 0.75 to 0.79. When Norman computed correlations on 
individual rather than group test-retest data, the highest 
correlation he found, out of 500 subjects, was 0.37. 

Heise (1969) investigated the assumptions behind 
analysis of semantic differential data. One assumption 


involves the bipolarity of the scales. Most scales commonly 
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used are a pair of polar opposites. Assumptions made when 
employing the semantic differential are that: (1) each 
member of the polar opposite pair is the same distance from 
the origin; and (2), members of different polar opposite 
pairs are at equal distances from the origin. According to 
Heise, the assumptiom of bipolarity of scales is not 
perfectly valid, but is close enough for most purposes. A 
second assumption involves the metric defined by the terms 
"slightly", "quite", and "extremely". Heise quoted an 
earlier study by Messick (1957) in which the method of 
successive intervals revealed correlations of from 0.98 to 
0.99 between actual and assumed metrics over nine common 
bipolar scales. This research was cited in support of the 
use of metric methods on semantic differential data. Based 
upon the work of Wiggins and Fishbein, Norman, and Heise, it 
seems reasonable to expect an acceptable level of stability 
and internal consistency on semantic differential 
instruments when used with the age groups in this study. 
However, interpretation of factors and generalization of 
results should be undertaken only with caution. 

As mentioned above, subjects repeated two pages of 
the semantic differential test. Correlations between their 
responses on these occasions were reported, in a manner 
Similar to the similarity judgment consistency measures, as 


semantic differential consistency measures. 
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Subjects! data were organized to produce a 
similarity matrix for each of the three concept relationship 
tasks. It is possible to calculate an overall subject 
consistency estimate by assuming that these matrices are, in 
fact, products of the same cognitive structure and should 
ideally be identical. Three correlation coefficients were 
produced for each subject by taking the three tests in pairs 
and correlating the vectors of the 105 Similarities 
produced. These three correlation coefficients were 
converted to Fisher's 727 (Ferguson, 1971), averaged, and 
converted back to correlation coefficients. This variable is 
reported as one of the generated variables of interest. It 
may be interpreted as reflecting both the attention paid to 
the tasks by the subject and the nature of the cognitive 
processes being tapped. Correlations for all subjects were 
low enough that the use of Fisher's Z to avoid ceiling 
effects in averaging was unnecessary. 

There is considerable conceptual difficulty with the 
problem of reliability of developmental level measures. 
Test-retest methods, because of the small number of tasks 
involved, are subject to strong memory effects. If a longer 
time interval is used, there is the real possibility of 
cognitive growth confounding the results. Measures of 
internal consistency, such as KR-20 and principal components 


analysis, assume the unifactor structure of formal 
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operations which is not a closed question. Nevertheless, 
both these calculations are reported as estimates of 
reliability. 

3:36 Field Dependence-Independence Test 

Hammond (1976) reported correlations between the 
Hidden Figures Test and the more established Embedded 
Figures Test (Witkin, 1950) in the order of 0.76 from 
Witkin, and 0.45 from other workers. Jackson et al (1964), 
in attempting to develop group versions of the Embedded 
Figures Test, produced three versions of tests similar in 
intent and content to the Hidden Figures Test. They reported 
correlations of about 0.60 between their tests and the 
Embedded Figures Test. While recognizing the inapplicability 
of KR-21 calculations to speeded tests, they reported KR- 
21's of the order of 0.75 for their various tests. Fleishman 
and Dusek (1971), using a sample of 85 enlisted army males, 
reported a test-retest coefficient over a few hours of 0.72. 
Boersma (1968), using a sample of 105 undergraduates, 
reported a test-retest coefficient over ten weeks of 0.63. 
He also reported KR-20's on these two administrations of the 
test at 0.79 and 06.87. Reliability of the test for this 
study will be estimated by split-half correlations between 
the odd and even items over both halves of the test. 
7 Cognitive Complexity Test 

Seaman and Koenig (1974) presented a summary of the 
problems involved in achieving consistency of ubaueee © of 


cognitive complexity. For example, in the Bieri or similar 
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systems, if dichotomous rather than Likert type scales for 
rating by the subjects are used, the analysis produces’ more 
ties and greatly affects the results. Tests of cognitive 
complexity ask for ratings on both positive people (those to 
whom the subject feels attracted), and negative people. 
Seaman and Koenig compared several methods of measuring 
cognitive complexity. They compared scores using Bieri's 
tied rating methods for all people rated, for positive 
people only, and for negative people only, along with scores 
proposed by Fiedler (1967) , involving the highest rating 
for a negative person and/or the lowest rating fora 
positive person, and an information density model proposed 
by Scott (1962). The Fiedler measures fared quite poorly in 
the comparison. Factor analysis showed that the Scott and 
Bieri methods loaded quite well on the same factor. A 
variation of the Bieri method, as used by Seaman and Koenig, 
has been chosen over the Scott method for this 
investigation. Scoring details will be provided in section 
3.5: 

Estimates of the reliability of the cognitive 
complexity measure used in this study have been obtained 
from test-retest data over one month from one of the grade 9 


pilot classes. 


3.4 Sample and Design 
Each of the concept relationship tests was paired 


with one of the personality variable tests for 
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administration on the same testing occasion. Pairing was 
done on the basis of time needed for administration. The 
pairings were: the Hidden Figures test with the similarity 
judgment test; the developmental level test with the 
constrained association test; and the cognitive complexity 
test with the semantic differential test. The personality 
test was always administered before the concept relationship 
test. 

The target population of the study is the population 
of high school science students. It is the intention of the 
investigator to generalize to individuals only. Since no 
generalizations to classes, grades, or schools are proposed, 
the restriction of the sample to two grades and four schools 
is not viewed as a limitation of the study. Restriction of 
the sample to one urban-suburban school system, however, 
should be viewed as a limitation on the generalizability of 
the results. 

With three testing sessions needed per group, it was 
planned to test six classes at each grade level, grade 9 and 
grade 12, each class receiving the testing sessions in one 
of the six possible orders. Such a large sample was deemed 
necessary because substantial sample attrition was expected 
due to the number of sessions needed to obtain complete 
data. Subjects were kept in the sample only if they 
completed all six tests, and if their I.Q. scores (for grade 
nine) and last year's achievement score were available. The 


particular grades used were chosen because the school board 
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administered system wide 1.0. tests in grade nine, and 
because a maximum possible spread between the two halves of 
the sample was needed to best investigate developmental 
aspects of the study. Biology students were chosen because 
they are usually more typical of the high school population 
than either chemistry or physics students. Also, biology 
students were easier to obtain due to larger enrolments in 
biology. 

AS anticipated, there was some difficulty obtaining 
a sample for the study. Most junior high schools in the area 
operate on 40 or 45 minute periods. Only a few had the 60 
Minute periods necessary. Further, the study was done in 
April, and many grade twelve teachers were reluctant to 
relinguish three hours of class time so close to the end of 
the term. Six grade nine classes were obtained, four in one 
school and two in another. Only five grade twelve classes 
were obtained, three in one school and two in another. The 
sample size desired was not substantially affected, as the 
two classes obtained from one school were team taught, 
having double the normal enrolment. For the senior sample, 
one of the six possible orders of testing was randomly 
discarded. 

Testing had been planned so that each class would be 
visited at the same time each week, for three weeks, with 
all administration done by the investigator. Due to the 
difficulties of Easter, field trips, etc., this was not 


possible. Gaps between testing sessions ranged from four to 
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nine days, with one of fourteen days. Thirty of the thirty- 
three sessions were administered by the investigator, and 
the remaining three by the investigator's thesis supervisor. 
A common script was followed, and all questions and 
irregularities recorded. The second investigator observed 
the principal investigator in two testing sessions to insure 
uniformity of administration. Sample attrition is summarized 
in Table I. Most of the subject errors involved 
misunderstanding the cognitive complexity test or producing 
pages of the semantic differential test with no variance. 
The "other”™ errors were spread evenly over the remaining 
four tests. Included in the "error" category are those 
subjects who quite obviously were not treating the tests 
seriously. The voluntary nature of the task combined with 
beautiful spring weather to account for the high absenteeism 
in the grade 12 sample. While it may be argued that the 
\ 
sample suffers from differential absenteeism, this is not 
viewed as a serious hindrance to the generalizations which 
it is possible to make from the study. Undoubtedly, the 
average ability of those making errors on the test is lower 
than that of those not making errors. However, since the 
personality variables of the sample are reported, and since 
it is not an intention of the study to generalize to 


"typical classes", there should be no major problem due to 


attrition. 
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3.5 Scoring Procedures 

3.51 Constrained Association Test 

Responses to the constrained association test were 
hand scanned, and the data cleaned up for machine scoring. 
In this process, spelling was corrected, extraneous words in 
the responses, such as "the" and "of" were eliminated, 
responses of more than one word were linked together to form 
a long pseudo-word, and the grammatical form of some 
responses was changed (e.g9., verbs were put in present 
tense). It was expected from pilot work that there would be 
many cases of responses which were different words, but 
which came from the same root, such as "conclude" and 
"conclusion". In order to have these words contribute to the 
relatedness indices, only the first six letters were 
compared by the scoring program. This eliminated most cases, 
except for "prove" and "proof". In such cases, "prove”™ was 
changed +o "proof". It is argued that such minor changes in 
the responses is in keeping with the spirit of the 
relatedness index. 

The technique used in this study to calculate the 
relatedness indices differs from the Garskof and Houston 
technique in three ways. First, the response is constrained 
by asking the subject to concentrate on the scientific 
meaning of the keyword. The nature of "scientific meaning" 
is, of course, left for the subject to implicitly define for 
himself. Second, Garskof and Houston took the RI's as the 


results of their experiment. In this study, the RI matrix is 
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treated only as the input to the scaling techniques 
described above. The indices have the status of similarity 
measures, to use the scaling terminology. 

The correction factor of minus one proposed by 
Garskof and Houston is not used in this study. Garskof and 
Houston reasoned that, since the first words on the two 
lists could never be the same, the index could not equal 
1.00 as a theoretical maximum. The correction factor allows 
the RI to be 1.00 in a case where two words elicit each 
other first as responses, and the rest of the words on the 
two lists are identical in nature and position. The 
correction factor has not been used in this study because it 
allows no distinction between short lists and long lists 
which are maximally related. Using the correction factor, if 
two words elicit each other, and then are followed in the 
response list by either one or ten more identical reponses, 
the RI will be 17.00. By not using the correction factor, the 
longer identical lists will produce a higher RI. The 
disadvantage of this procedure is that the theoretical 
maximum of the relatedness index is now less than one. 
Analysis shows that, for the usual range of values found, 
the difference in calculation makes no practical difference. 
For example, in the example relatedness index calculation of 
section 2.41, the value is 0.444 using the correction, and 


0.439 without it. 
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3252 Similarity Judgment Test 

For the similarity judgment tests, scoring was done 
by laying a clear plastic sheet with vertical divisions on 
it over the response pages. If a subject missed one 
judgment, the value of "5" was arbitrarily supplied in order 
not to lose the subject from the sample because of a small 
omission. If he missed two judgments, he was dropped from 
the sample. Only three of the 132 subjects have had a 
response supplied in this manner. It is doubtful that such 
tampering would change even individual results by any 


noticeable amount. 


In the semantic differential test, there seemed 
considerable evidence of subjects refusing to take the 
requested judgments seriously. This would account for the 
dozen subjects who simply ran down the pages marking the 
same point on every scale. These subjects were easily 
detected when an attempt was made to form a correlation 
matrix of their scores. Such subjects, as indicated in 
Table I, were dropped from the sample. 

3-54 Developmental Level 

The developmental level test was scored as follows. 
Fach item was worth two points. Item one was the volume 
displacement task, and was worth either 0 or 2 points. Under 
the testing conditions for item 1, there was no opportunity 
to award part scores. All other items were scored 0, 1, or 
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In item two, the hamburger question, if a subject 
produced an incomplete and unsystematic response, he scored 
zero. If he produced a partly systematic but incomplete 
response, or if he filled the page producing permutations on 
the order of a few of the correct responses, or if he 
happened upon a complete solution in what appeared to be a 
random manner, he scored one. If he proceeded in a 
systematic manner, and showed evidence that he was aware of 
forming combinations of more than two ingredients at a time, 
he scored two even though he may have missed one or two of 
the fifteen combinations. As a point of interest, only one 
or two subjects mentioned the possibility of the sixteenth 
hamburger, the plain one. 

Item three was the screws puzzle. If a subject got 
none or one of the three parts, he scored zero. For two 
correct, he scored one, and for all three, he scored two. 
This procedure assumes that getting only one of the three 
parts is luck rather than skill. 

Item four was the belts question. There were eight 
parts to this item, with scores ranging from zero to eight. 
In the absence of any evidence on the behaviour of this iten 
with respect to developmental level, a completely ad hoc 
scoring procedure was devised. Since the majority of 
subjects found the item very easy, a method was desired to 
give a satisfactory distribution between scores of zero, 
one, and two. On the eight point scale, five or less 


received zero, six received one, and seven or eight received 


icukitve 6 Ti \notened@aepmudieet edt ave, wand 
Ketest 24 .eRsodt 6? Si Wpoteyr dn Ban “re tawoipad ae basubowq 
atataneont td arsemeteym. giltraq © basinbecemeied 21\.070F 
go tat pee THe oni nUbOst epsq 44% bullied. oa $i 10 ,aenogae7 
14 Yt JO .nehooqzes Anegtes add Fo wo? #§ to wtebso eat 
» ef of berpeaan tad at aoktofoe etalun99'6 noqu beneadsd 
& wi foboenogn ef FF sane | hetooe od ,te0nsn 8 wobast, 
r een ape of ted? eonebive beyods bas ,290n68 Zitamoteye 
wit ow *& ptnethoapad owt ned? a1ow Yo anottetbdaoy. panier? 
+o uve Jo eno torake ick wi dipvod> save owt hbe1tooa, ad 
ano vfao ,taete¢nt Yo tatoo #6 2A .2aottanidson avotth? off 
‘enaadxde od? Jo vtifidiaeed eft bonodtaom etoetdua ows Ze 
.onn mielq oft .toptedasd 
ton tsetdns 6 97 se hspaq eyeroe et sew eotdt antl . 
nee an? .ntew hetas2ced qattag cotdt ad? Ja 40° 20> Seen 
owt beiooe od ,99adt Tin to? ba6 ,ono herooe ed , tae tI09 
t7' oft de ono yigo pili tap. felt cenizes saybooorq eidT 
«LLite oad rodter Youll ai etasqg 
tenis pw otort .metteerp atied oft 2ew 190% gadt 
_tipie o¢ opay colt paipaps ae7ooe dtiv .aert eid? oF atmsq 
aeti adidt to reotveded af% fo snnebive yas to songads odd al 
sof bg vietefamoo «6  fevel [stnanqolavebh oF jasqeet tiv 
ty oy *tbtoter ont) 6spote eborivel ere sivbes0719 - ‘pakzegs. 
o* hetéiteb atv bodtem 6 ,yasp YIev aot add bauod edvebden 
orm Yo gatose nee ted ner paper: 


97 


two. 

On the total developmental test, a subject could 
score from zero to eight. This scale was divided into equal 
thirds. Scores of zero, one, or two were grouped as 
concrete, six, seven, or eight as formal, and three, four, 
or five as borderline. This system allowed a subject to miss 
one task completely or be borderline on two tasks and still 
be graded as formal operational. At the other end of the 
scale a subject could be borderline on two tasks, or get 
only one completely correct and still be classified 
concrete. The system tended to err on the side of putting 
subjects in the middle group. This scheme produced a 
reasonable spread of developmental level, which was roughly 
in keeping with the distributions reported by other workers 
(c.f., Lawson and Renner, 1974>- 1975). There was not, 
however, as much difference between the two grades as might 


be expected. 


scheme supplied (French et al, 1965), both with and without 
a correction for guessing. 
3-56 Cognitive 
The cognitive complexity test produced a six by six 
matrix of ratings of acquaintances on characteristics. The 
scoring procedure looks for ties in the ratings of the same 


person on different characteristics. With six 


characteristics, the maximum number of ties is fifteen per 
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person, for a total of 90. Scores of complexity were the 
number of ties subtracted from 90. 


3-57 Expurgation of Data 


In studies of this type, caution must be taken to 
identify those subjects who refuse to take the tests 
seriously and produce meaningless or frivolous results. Hand 
checking of the constrained association data uncovered two 
subjects who were obviously playing jokes. Random Or 
systematic pattern production, without regard for the task 
in the semantic differential test resulted in the dropping 
of 13 subjects from the sample. Any such subject remaining 
would, on the individual scaling analyses, produce patterns 
so isolated and unique that their patterns would not be 
taken as honestly reported cognitive structures. Such 
patterns would almost certainly be interpreted as error in 
the data. There is no evidence that those subjects who 
remained in the sample did not treat the tests seriously. 

It is also important to watch the individual 
administrations of the tests for Variation. Certain 
questions were expected from pilot studies, and answers were 
scripted. Occasionally other questions arose and were dealt 
with. Wone of the administrations was substantially 


different from the others. 


3.6 Analytic Procedures 
The following considerations apply to the formation 


of the similarity matrices. Responses to the constrained 
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association task were used to produce a set of relatedness 
indices which were used directly as similarities. As 
calculated by Garskof and Houston, RI's have a range of 0.00 
to 1.00. The method of calculation used in this study and 
the maximum value are slightly different, as has been 
discussed in section 3.61. Responses to the similarity 
rating task produced a set of dissimilarities ranging from 1 
(most similar) to 9 (least similar). These values were 
subtracted from 10 tc produce a set of similarities. The 
semantic differential results were used to produce a set of 
correlations among the patterns produced for each of the 
concepts rated. The theoretical range of the semantic 
differential correlations was -1.00 to 1.00. 

The Similarity judgment data are affected by (a) 
response sets, giving varying mean Similarities produced by 
different subjects, and (b), the tendency to exaggerate or 
attenuate similarities, producing varying variances across 
subjects. Since these possibilities had the potential to 
confound the analysis, and since they were not considered to 
be of theoretical interest, the Similarity judgment data 
were standardized to a mean of 10 and a standard deviation 
of 2.00 within each subject. The confounding thus avoided 
would have allowed those subjects with the largest variances 
in similarity to dominate any group average of which they 
were a part. 

The semantic differential data are also open to the 


possibility of correlations amcng concept response patterns 
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being enlarged due to response sets. For example, a subject 
might tend to view most concepts presented as "good" or 
"weak". His inability to make distinctions within each of 
these dimensions would increase his average correlation, and 
allow him to dominate a group average. To avoid this 
problem, the semantic differential correlation matrices were 
also renormed to mean 10.0 and standard deviation 2.00. 

In the constrained association test, the tendency 
for a subject to use a relatively small pceol of words from 
which to draw his responses, especially to use some of the 
concepts as responses to other of the concepts, would 
produce a large average relatedness index compared to that 
of a subject who responded from a larger pool of words. This 
tendency was judged to be of sufficient theoretical interest 
that the relatedness index matrices were analysed both as 
calculated and as renormed to mean 10.0 and standard 
deviation 2.00. Thus, each subject produced a total of four 
matrices for further analysis: relatedness indices scaled 
and unscaled, similarity judgment, and semantic 
differential. For brevity, the last two mentioned tests will 
not be referred to as scaled, since they are never discussed 
in the unscaled form. The matrices will be referred to below 
by these terms. 

Considerable research has been carried out (Klahr, 
1969; Wagenaar and Padmos, 1971; Isaac and Poor, 1974; 
Clark, 1976) on the significance of the stress function for 


interpretation of results. Klahr (1969) did a Monte Carlo 
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study of the Kruskal scaling technique. His results showed 
that the method had the ability to determine structure for 
points when, in fact, no such structure existed. Since 
scaling procedures always produce numbers, the experimenter 
must decide subjectively the difference between 
interpretable and uninterpretable results. Klahr's results 
emphasized the danger of overinterpreting results 
particularily where a small number of data points are used. 
To combat this overinterpretation, he suggested that the 
range of acceptable stress values should be considerably 
smaller than Kruskal had originally proposed. Kliahr also 
noted that the resulting geometrical mapping was unreliable 
for fewer than ten points. 

Kruskal also proposed a second "8S" function, which 
he called Stress 2 (see Kruskal and Carmone, 1970). Kruskal 
and Carmone have produced revised estimates ot what 
constitutes "good" or "poor" stress for different numbers of 
data points and for both Stress 1 and Stress 2. According to 
Kruskal and cCarmone, stress 2 values are generaily about 
twice the size of Stress 1 values. The Monte Carlo studies 
(Wagenaar and Padmos, 1971; Isaac and Poor, 1974) reported 
which investigate the effects cf error on stress all present 
their results in terms of stress 1. Since these studies are 
relevant to a discussion of the error in the present data, 
ali stress values are reported as “stress 1. Using the 
Schonemann and Carroll (1974) non-metric fit procedure as a 


measure of congruence, scaling the same data using stress 1 
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or 2 produces soluticns which correlate at least 0.95. 
Considerable WOrk has been done on the 
interpretation of multidimensicnal scaling results. Wagenaar 
and Padmos (1971), Isaac and Poor (1974), and Spence and 
Graef (1974) have all carried cut Monte Carlo studies on the 
effects of error on the interpretability of scaling results. 
The studies all involve generating results for several 
different numbers of points in several different numbers of 
dimensions when the true dimensionality of the solution is 
known. The Similarities are then perturbed by several known 
amounts of errcr, corresponding to subject error in 
estimating the similarities. The effect of these various 
conditions on the stress values of the final solutions can 
then be observed. Spence and Graef have produced results for 
configurations of 12, 18, 26, and 36 points, but only 
present detailed results for the 36 point case. Wagenaar and 
Padmos dealt with 8, 10, and 12 point configurations. Isaac 
and Poor dealt with, among others, configurations of 12 and 
16 points. On the basis of Isaac and Poor's study, the data 
of the present study are best interpreted as having a real 
dimensionality of three dimensions, and as containing about 
40 percent error. The stress 2 values of about 50 percent 
are not unreasonable, and are well below the values reported 
for 16 random points. An attempt at extrapolation from the 
Wagenaar and Padmos data would indicate even less noise in 
the data of this study than that suggested by the Isaac and 


Poor results. 
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All three Monte Carlo studies indicate that an elbow 
in the stress plot is only significant for the zero error 
case. Isaac and Poor's calculation of Kruskal'’s metric 
determinancy function indicates that the distances produced 
by the scaling procedures on the present data would 
correlate about 0.75 with the criginal dissimilarities. 

Cohen and Jones (1974) reported a Monte Carlo study 
in which a parameter was introduced to simulate a subject's 
attention wandering among the dimensions of perception 
during a rating test. Under conditions of 15 points in three 
dimensions, with 50 percent error in judgment and simulated 
random attention to only two cf the three dimensions of 
judgment at any one time, Cohen and Jones still achieved 
0.75 correlation between estimated and actual distances. 
They conclude, on the basis of other results, that other 
Monte Carlo studies {(e.g., Wagenaar and Padmos, 19/1) have 
underestimated the sericusness of "underdimensioning” a 
solution. They recommend that a dimension only be discarded 
if the investigator is certain it ccntains only error. They 
Make no mention of the criterion of interpretability for 
deciding the dimensionality of the solution. Although there 
have been no responses fcund in the literature to Cohen and 
Jones, the ignoring of an "interpretability" criterion seems 


a serious flaw in their conclusions. 
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3.7 Summary 

This chapter has presented details of test 
development, sccring procedures, experimental design, and 
sample selection. Instrument reliability was discussed, and 
the analytic procedures explicated. The results of the study 


are presented in the chapter which follows. 
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CHAPTER 4 


RESULTS AND DISCUSSION 


4.1 Introduction 

This chapter presents the results of the study 
together with a discussion of their meaning as related to 
the hypotheses of the study. Section 4.2 presents data which 
facilitates interpretation of the non-metric orthogonal 
rotation goodness-of-fit measure ({Schonemann and Carroll, 
1970; tLingoes and Schonemann, 1974) used in order to 
estimate the fit of one matrix of scaling results to 
another. The next section reports data pertaining to the 
reliability estimates of the instruments used in the study. 
Section 4.4 contains the results of Kruskal scaling 
procedures as applied to the group average data for each 
grade obtained from each of the concept Similarity 
instruments. Section 4.5 reports the Tucker scaling results 
for each instrument for each grade, the formation of 
subgroups based upon the Tucker results, and subsequent 
Kruskal scaling of the subgroup average data. Section 4.6 
discusses the personality variables, and differences in 
these variables among the subgroups formed by the Tucker 
scaling process. Sections 4.7 and 4.8 parallel 4.5 and 4.6, 


but for the Carroll analysis. Section 4.9 summarizes the 
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chapter. For compactness of presentation, +he short forms 
summarized in Table II will be used as headings for many of 


the tables below. 


4.2 The Goodness-of-fit Measure 

The purpose of this section is to provide data which 
will help the reader understand the meaning of the goodness- 
of-fit measure used fer comparison of scaling configurations 
in the present study. The measure will be related to the 
more common Pearson product-moment correlation coefficient. 
Schonemann and Carroll (1970) have presented a nonmetric 
version of the orthogonal Procrustes rotation. The goodness- 
of-fit measure which they produced was improved by Lingoes 
and Schonemann (1974). The Lingoes and Schonemann measure of 
fit has both a symmetric version, in which the rotation of A 
to B gives the same fit as the rotation of B to A, and a 
non-symmetric version, in which this is not the case. Both 
versions are bounded by 0.00 and 1.00. 

Lingoes and Schonemann argue that the non-symmetric 
version, S, is the matrix analogue of a coefficient of 
alienation, 1.00 minus a squared correlation coefficient. In 
investigating the interpretation of their goodness-of-fit 
measure, the following calculations were performed. From the 
pool of data generated from the main investigation, pairs of 
matrices were selected whose symmetric goodness-of-fit 
measures spanned the range from 0.17 to 0.96. All the 


matrices were 15 by 3 mappings of the concepts under 
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TABLE II 
SUMMARY OF SHORT FORMS USED IN TABLES 

Constrained Association-Unscaled Grade 9 ...CAU9 
Constrained Association-Unscaled Grade 12 ..CAU12 
Constrained Association-Scaled Grade 9 .....CAS9 
Constrained Association-Scaled Grade 12 ....CAS12 
Si Si barity-aludgaent Grade 9 w.nssse so wine a eistlo 
Siasiarity aucguent Grate 12 «cw< anes sasmmeeied 12 
Semantic Differential Grade 9 ...2csccsc es 002 SDI 


Sesantic: Ditterential Grade 12 «ssiece weweacmsDiz 
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investigation. Based upon the value S, a value r was 
calculated using the formula supplied by Lingoes and 
Schonemann. For each pair of matrices, A and By a 
correlation coefficient was calculated between the values of 
B, and the values cf BHAT, the best rotation of A to fit B. 
This correlation coefficient was calculated by reforming the 
45 entries in each matrix into a vector, and correlating the 
values in the vectors. As can be seen from Table III, the 
correspondence between the r derived from S and the 
correlation coefficient is virtually perfect. It should he 
noted that, while these calculations offer a benchmark by 
which to interpret the goodness-of-fit measure, they do not 
provide any criterion for judging the statistical 
Significance of the S and r values. For the convenience of 
the reader, both the symmetric coefficient of alienation, SS 
and the derived xr value will be reported for any matrices 
compared in the remainder of this report. The symmetric 
coefficient of alienation, SS, is the square root of the 
non-symmetric version, S. 

To assist further in interpretation of the S values, 
Figure 3 shows the plots for selected dimensions of two 
pairs of matrices in three dimensions. The arrows go from 
the position of each point in matrix BHAT to the 
corresponding position in matrix B. It can be seen from 
Figure 3 that an SS value of 0.51 (r=0.86) corresponds to a 
pair of matrices with the majority of points shifting only a 


small amount. At this level of shift, however, some possibly 
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TABLE III 


RELATIONSHIP BETWEEN GOCDNESS-OF-FIT FUNCTION 
AND CORRELATION COEFFICIENT 


frial SS xr From S$ Corr Froa 


Matrices 
1 0.77 0.99 0.99 
2 0.38 0.92 0.92 
3 0.51 0.86 0.86 
4 0.62 0.78 0.78 
5 Oo. 77 0.64 0.64 
6 6.85 0.253 0.52 
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interpretable trends, such as the shift in points in the 
lower left hand quadrant, can be seen. Further, an SS _ of 
0.96 (r=0.28) corresponds te large shifts in almost all 
points. Such ubiquitous shifting would lead to substantial 
differences in interpretation of the meanings of the 


dimensions. 


4.3 Reliability of Instruments 

Reliability estimates for the instruments used in 
the study are summarized in Table IV. 

There are five estimates of reliability of the 
constrained associaticn test based upon the test-retest data 
collected from a sample of grade nine subjects not part of 
the main study. The first estimate listed was obtained by 
correlating the two vectors of 105 similarities produced by 
each subject on the two occasions and averaging the 
individual correlations over the 18 subjects. Since 
reliabilities for individuals are rarely reported in the 
literature, the next two estimates are reported for 
comparison purposes. The similarities were averaged over the 
subjects, and the two vectors of 105 averaged similarities 
produced by the group on the two occasions were correlated. 
This calculation was performed on both the scaled and 
unscaled similarities. Since the norming was done  hbefore 
averaging, it is coincidence only that the correlation 
obtained is exactly the same for the scaled and unscaled 


data. 
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TABLE IV 


SUMMARY OF RELIABILITY ESTIMATES 


_———— a mr ee ee ee er re ee ee ee a eee ee eae ee ee ae i ee 


Constrained Association Test 

Test-Retest (N=18) 
Average Indiv. Corr. 0.33 
Group Corr.-Unscaled 0.76 
Group Corr.-Scaled 0.76 
Kruskal Goodness-of-fit Scaled Bi | 
Kruskal Goodness-of-fit Unscaled 0.70 

Personal Consistency 
Grade 9(N=78) 0.46 
Grade 12(N=54) 0.42 


Similarity Judgment 
Personal Consistency 
Grade 9(N=78) 0.56 
Grade 12 (N=54) Get 
Semantic Differential 
Personal Consistency 
Grade 9(N=78) 0.60 
Grade 12 (N=54) 0.64 
imilarity Measures 
Average Consistency 


jw 


Grade 9(N=78) 0.20 
Grade 12 (N=54) 0.16 
Developmental Level Test 
KR-20 
Grade 9({N=78) 0.45 
Grade 12 (N=54) 0.50 
Prin.Comp.-Variance on First Factor 
Grade 9{N=78) 0.38 
Grade 12(N=54) 0.43 


Hidden Figures Test 
Split Halves-Correction for Guessing 


Grade 9(N=78) a ae 

Grade 12 (N=54) 0.78 
Split Halves-No Correction 

Grade 9(N=78) 0.76 

Grade 12 (N=54) 1 es 


Cognitive Complexity 
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Test-Retest -0.11 
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The group average data produced by the group on the 
two occasions were analysed by Kruskal scaling, and rotated 
to a best fit with each other. The SS values corresponding 
to the r values in Table IV are 0.85 and 0.71 respectively. 
A possible explanation for these two reliability estimates 
being so different might lie in the difference between the 
average number of responses and the average relatedness 
indices produced on the two occasions. For the pretest, 
these values were, respectively, 80.7 and 0.07, while for 
the posttest, they were 101 and 0.10. The effect of the 
rescaling procedure would be different in the two cases, 
adversely affecting the test-retest reliability. Such an 
increase in fluency on the retest might be attributed to 
familiarity with the instrument. The two testing occasions 
were one month apart. The goodness-of-fit r for the unscaled 
data can probably be taken as a better estimate of 
reliability than can that for the rescaled data. 

No similar test-retest data are available for the 
Older group. Since the reliabilities for other measures 
reported in Table IV show no substantial differences between 
the two grade groups, it may be reasonably inferred that 
test-retest data for grade twelve subjects, if they were 
available, would not be substantially different from that of 
the grade nine subjects. 

The personal consistency estimates for all three 
Similarity rating tests were calculated as described in 


Chapter 3. These personal consistency estimates are judged 
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to be satisfactory. 

The average consistency measure produced by each 
subject over the three similarity rating tests produced 
quite a low consistency estimate. As noted in chapter 3, 
such an estimate assumes that the three tests are tapping 
identical cognitive structures. Thus, the estimate is 
confounded by having to make the same type of assumption as 
made for the estimates of reliability of the developmental 
level test. That is, it is not clear whether the correlation 
coefficient is measuring convergent validity or consistency 
reliability. Keeping this doubt in mind, it may be 
hypothesized that the consistency estimate is not out of 
line with the only other individual reliabilities reported, 
those for the test-retest on the constrained association 
test. 

As expected, the available methods of estimating 
reliability of the developmental level test do not produce 
very high reliability estimates. Possible reasons for these 
results have been discussed in Chapter 3. 

The Hidden Figures Test produced reliability 
estimates in keeping with expectations based upon values 
reported in the literature. The correction for guessing 
seems to have no effect. As an aside, the correlation 
between the corrected and uncorrected scores was 0.99. 

The reliability estimate for the cognitive 
complexity test reveals it to have an unacceptable error of 


measurement. This can possibly be explained by the size of 
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the matrix used. Seaman and Koenig (1974) discussed larger 
matrices (8 by 8 and 13 by 13) than that of the present 
study. It is possible that, with only six persons to rate, 
the measure was too dependent upon the persons chosen. For 
the posttest, subjects were asked to rate the same persons 
if they could remember who they were, or to rate different 
persons if they could not. Fourteen subjects reported that 
they used different people, three that they tried to 
remember who the persons rated on the first test were, but 
were not very confident that they had been successful, and 
only one subject indicated that he was certain he was rating 
the same persons. Considering the difficulty which grade 
nine subjects had with the directions and the length of time 
needed to complete a six by six matrix, it is difficult to 
see how, given the time constraints of the study, the 
judgment matrix could have been enlarged. Given that the 
grade twelve subjects did not experience even a fraction of 
the difficulty with this test experienced by the grade nine 
subjects, it might be hypothesized that a similar test- 
retest for grade twelve subjects wonld yield higher 
reliability estimates in this particular test. It is 
unfortunate that subjects were not available for such a 


testing. 


4.4 Group Average Results 
Kruskal scaling was performed on the group average 


results. The stress values for solutions in from six to two 
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dimensions are reported in Table V. For brevity, the data 
sets will be referred to by number, as given in Table V. As 
mentioned above, the purpose of reporting group average 
solutions is to assess the overall dimensionality of the 
cognitive structures under investigation. Using the "elbow" 
criterion as a rough guide, data sets 3,5, and 8 (CAS9, SJ9, 
and SD12) seem to require a three dimensional solution, data 
sets 2, 4, and 7 (CAU12, CAS12, and SD9) seem to require a 
four dimensional solution, while the decision in cases 1 and 
6 (CAU9 and 5$J12) is not as clear. As noted in chapter 3, 
Wagenaar and Padmos (1971) maintain that the elbow criterion 
is applicable only in cases of error free data. 

A decision was taken to report only three 
dimensional solutions in all cases for the following 
reasons. First, the data are certainly not errorless, and 
thus, according to the criteria of Isaac and Poor (1974), 
the stress values for the three dimensional solutions are, 
in all eight cases, acceptable for a situation in which the 
data contain errors in the order of 40 percent of the 
Similarity values. In light of the consistency and test- 
retest reliability estimates reported for the data in the 


previous section, the assumption of such an error level in 


the data seems reasonable. Second, the study is an 
investigation of individual differences in cognitive 
structures. If subjects are regrouped according to a 


criterion of similarity, such as are produced by Tucker or 


Carroll scaling, it seems reasonable to expect a subgroup to 
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1.Constrained Association 
Unscaled Grade 9(N=78) 


2.Constrained Association 
Unscaled Grade 12 (N=54) 


3.Constrained Association 
Scaled Grade 9 (N=78) 


4Y.Constrained Association 
Scaled Grade 12(N=54) 


5.Similarity Judgment 
Grade 9{N=78) 


6.Similarity Judgment 
Grade 12 (N=54) 


7.semantic Differential 
Grade 9(N=78) 


8.Semantic Differential 
Grade 12 (N=54) 
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be more homogeneous than the total sample. A possible reason 
for requiring a four dimensional solution would occur in a 
Situation in which part of the sample based judgments on, 
for example, dimensions 1, 2, and 3, while another part of 
the sample based judgments upon, say, dimensions 1, 2, and 
4. Analysis of subgroups in such a case would produce three 
dimensional solutions, with two of the dimensions similar, 
but the third one different. Such an occurence in the 
present study cannot be ruled out. Third, based upon 
expectations from the pilot work, +hree dimensional 
solutions would be more readily interpretable than four 
dimensional solutions in this study. Fourth, data sets 2, 4, 
6, and 8 from Table V are grade twelve data. Because of the 
smaller sample size in grade twelve, these data sets may be 
expected to contain more error of measurement than the 
larger grade nine sample data. As will be reported in detail 
later, this point is supported by stress values for scaling 
solutions of smaller subgroups in general. This would 
account for the relatively high stress valves found in the 
CAUI2,  .CASI2, ann Sd12 data. Faftth, and in partial 
contradiction to the previous statement, a case can be made 
for three dimensional solutions based upon the reliability 
measures of Table IV. Based upon the personal consistency 
measures for the three tests, the grade nine sample had less 
error in the constrained association and similarity judgment 
tests, while the grade twelve sample had less error in the 


semantic differential test. Although the significance level 
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of the consistency measure cannot be estimated from the 
data, and the difference between the grades is small, the 
trend indicated would predict higher stress values for cases 
2, 4, 6, and 7, and lower stress values for cases 1, 3, %- 
and 8. Sixth, as will be reported in section 4.7, one of the 
results of the Carroll scaling analysis is a correlation 
between the scaling solution and the raw data input for each 
subject. The average correlation between data and solution 
did not improve sufficiently when moving from three to four 
dimensions to warrant reporting of four dimensional 
solutions for the Carroll scaling analysis. 

The dimensions revealed by Kruskal analysis are 
arbitrary, and therefore solutions for the different data 
sets cannot be compared directly with each other. The 
solutions must be rotated to some common target before 
comparisons are possible. Since Carroll and Chang (1970) 
Maintain that the dimensions found in their analysis are 
meaningful, one of the group average results from the 
Carroll analyses was arbitrarily chosen as the ccmmon target 
to which to rotate the Kruskal results of the eight data 
sets. The solution chosen was the grade nine similarity 
judgment solution. Table VI reports the three dimensional 
Kruskal solutions for the eight data sets rotated toa 
common target. Again, because of the arbitrary nature of the 
orientation, it should be noted as an aside that no comment 
can be made concerning the relative importance of each 


dimension based upon the size of the average loading on each 
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dimension. 

In chapter 3, it was noted that the dimensions 
interpreted as time and certainty were confounded in the 
pilot work. This problem, as expected, has persisted into 
the main study. In attempting to label the different 
dimensions, relatively subtle differences in loadings of the 
concepts have been focused upon in order to create 
distinguishable labels. For example, the main distinguishing 
feature of a dimension labelled "temporal" rather than 
certainty is the presence of a high loading for the concept 
"conclusion". Similarly, the distinguishing feature of a 
dimension labelled "certainty" rather than "creativity" is 
high loadings for the concepts "puzzle" and "question" 
rather than "imagination" and "hypothesis". As will be seen, 
these various dimensions are rarely clearcut. The terms 
"law", "fact", “evidence", and "proof" tended to cluster 
quite closely together on most dimensions. Thus, they were 
not useful for distinguishing dimensions. The concepts at 
the opposite ends of the continua from these terms, e.g., 
those which were "less certain", "earlier", or "more 
creative", were usually the best basis for labelling the 
dimensions. The label "theoretical vs. practical" was 
employed on those occasions when "law" or "theory" was close 
to "hypothesis" and opposed to "conclusion", "fact", or 
"discovery". A summary of interpretations given the Kruskal 
configurations is presented in Table VII. 


For the unscaled grade nine constrained association 
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SUMMARY OF DIMENSION INTERPRETATIONS POR KRUSKAL 
SOLUTIONS OF FULL GROUP DATA 
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results (CAU9), the first dimension has been labelled 
"certainty". The concepts "puzzle" and "question" are 
opposed to the cluster of "law", "fact", "evidence", and 
"proof". The concepts "conclusion" and "explanation" do not 
load highly on this dimension. Dimension two, which clusters 
"hypothesis", “imagination", and "interpretation" opposite 
"discovery", “law", and "proof" has been labelled 
"creativity". The third dimension has been labelled 
"temporal" because of the high loading of the concept 
"conclusion". "Conclusion " has been chosen as a relatively 
Clearly "temporal" rather than "certainty" concept. 

In the second set of results, the grade twelve 
unscaled constrained association (CAU12), the first two 
dimensions have been labelled "certainty" and "creativity", 
but the third dimension has been left unlabelled hecause of 
the puzzling coupling of "conclusion" with "imagination" and 
“hypothesis” in opposition to "discovery", “evidence”, and 
"explanation". 

The effect of the rescaling of the grade nine data, 
CAS9, is most noticeable in the third dimension, which has 
been labelled "theoretical-practical" rather than 
"temporal". The first two dimension labels remain the same 
as for the cCAU9 data. The third dimension of CAU9 was 
labelled as it was because of the coupling of “law" with 
"theory", in opposition to "discovery" and "conclusion". 

The rescaling of the grade twelve data (CAS12) had 


no effect on the labels of the first two dimensions, but the 
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third dimension, although quite different from that of the 
CAU12 data, still remains puzzling and unnamed. 

The grade nine similarity judgment dimensions were 
labelled "certainty", "creativity", and "certainty", 
respectively. Dimensions one and three are correlated 0.54. 
No other label was deemed appropriate for dimension three. 
The corresponding dimensions for the grade twelve sample 
(SJ12) were labelled "certainty", "creativity", and 
"theoretical-practical". 

The semantic differential data seemed to require 
considerable stretching of the rules established for 
labelling the first six matrices reported. For example, in 
the grade nine data (SD9), the first dimension was labelled 
certainty despite the fact that "question" did not occupy, 
as in the first six data sets, its usual position quite 
close to "puzzle". Similarly, dimension two was labelled 
"creativity" despite the absence of a high loading for 
"imagination". The third dimension was not labelled. In the 
grade twelve data (SD12), the first dimension was labelled 
certainty, but the second dimension seemed closer to the 
criteria for "theoretical-practical" than "creativity" 
because of the large loading for "experiment". Again, it was 
not possible to name the third dimension. 

In summary, a few limited generalizations about the 
concept perceptions as indicated by the Kruskal solutions 
can be made. First, the dimensions of "certainty" and 


"creativity" (with one exception) seemed relatively stable 
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throughout the eight sets of data. The third dimension 
varied enough that it seemed to require different labels for 
the different sets of data. There are no obvious patterns 
across either similarity measures or grades for the meanings 
assigned to dimension 3. The concept "imagination" was, in 
general, not as isolated for the grade twelve sample as it 
was for the grade nine sample. Because of its relative 
proximity to "hypothesis" and "theory", the concept 
"imagination" could perhaps be viewed as more accepted as 
part of the scientific reasoning process. This in turn could 
reflect a more mature appreciation of the role of 
imagination in the work of the scientist. The concept 
“discovery” appeared not to be treated in the sense of 
discovery of a theory, but more in the sense of invention. 
When it played a significant role in the definition of a 
dimension, a was usually grouped with "law" and 
"conclusion" rather than "theory". Finally, the four 
concepts "law", "fact", “proof", and "evidence”™ were quite 
consistently clustered together. A main purpose of this 
study was to search for construct validity of cognitive 
structure. It was hypothesized that such validity would be 
achieved by production from the data of scaling 
configurations interpretable in the context of the 15 
example concepts chosen for examination, and by convergence 
of these configurations produced from the data of different 
Similarity rating tests to the same solution. The results 


presented in Table VII and their discussion have _ provided 
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the first part of this evidence for construct validity. The 
results presented next address the second half of the 
construct validity of cognitive structure, discriminant 
validity. 

Further detailed analysis and reporting of the 
cognitive structures of the various subgroups formed in this 
study could amount to information overload for the reader, 
and because of the volume of data involved would distract 
from discussion of the more important questions of 
convergence and individual differences. Reporting of the 
structures will be confined to the use of the qoodness-of- 
fit measure unless, in a particular case, a point pertinent 
to the main hypotheses (convergence and individual 
differences) of this investigation can be made by reference 
to the concepts themselves. The example concepts chosen are 
not themselves central to the hypotheses of the study. 

Results pertaining to the convergent and 
discriminant validity of the three tests are summarized in 
Table VIII. As stated above, "SS" is the symmetric goodness- 
of-fit measure between the rotated matrix and the target 
matrix, and "r" is the corresponding correlation {see Table 
III). The top section of the table reports the effect of 
rescaling the constrained association data to minimize 
effects of differences in fluency of response. These data do 
not directly address the issue of convergence, but are of 
interest in terms of the behaviour of the constrained 


association data. Although, as reported in the previous 
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TABLE VIII 


GOODNESS-OF-FIT MEASURES FOR KRUSKAL SOLUTIONS 
OF FULL GROUP AVERAGE DATA 
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section, the labels for the dimensions were not consistent 
for the scaled and unscaled cases, the goodness-of-fit 
measures indicate little difference in the matrices. It can 
be tentatively concluded that, for the full group average 
data, rescaling has little effect. 

The next two sections of Table VIII summarize the 
evidence for the convergent validity of the instruments. As 
such, they address one of the major hypotheses of the study. 
For both the grade nine and grade twelve data, there seems 
to be good convergence between the constrained association 
and the similarity judgment results, with substantially 
poorer convergence for the semantic differential results 
with the other instruments. The poorer convergence of the 
grade nine semantic differential results compared to that of 
the grade twelve classes may be attributed to the greater 
difficulty experienced by the grade nine subjects on the 
semantic differential test. 

The bottom section of Table VIII summarizes data 
which indirectly addresses the discriminant validity of the 
instruments. The assumption which must be made in 
interpreting these goodness-of-fit measures as discriminant 
validity indices is that the cognitive structures of grade 
nine and twelve subjects are actually different with respect 
to these concepts. According to Campbell and Fiske (1959), 
the goodness-of-fit measures in the bottom section of the 
table should be substantially lower than those in the middle 


two sections in order to provide positive evidence for 
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construct validity. As can be seen, this is not the case. 
However, aS pointed out in chapter 3, we have no way of 
determining the extent to which the cognitive structures of 
grade nine and twelve subjects should differ. Because 
subjects separated by grade may not differ substantially in 
cognitive structure does not rule out the possibility that 
there exist other variables along which subjects may be 
grouped in order to isolate differing cognitive structures. 
The individual scaling sections of this investigation are 
addressed to this issue. 

The convergence and discrimination among the scaling 
configurations have addressed the hypothesis of construct 
validity of cognitive structure. Convergence bet ween 
constrained association and similarity judgment was quite 
high, with the semantic differential instrument yielding 
solutions which did not converge as well. It may be 
hypothesized that the semantic differential instrument did 
not provide enough bipolar scales to cover all the 
dimensions of semantic space along which subjects were 
rating concepts as revealed by the other two instruments. 
Whether or not the instrument could be modified to provide 
better convergence with the other two instruments is a 


matter for further investigation. 


4.5 Tucker Scaling Results-Cognitive Structures 
The Tucker scaling procedure consists of a principal 


components analytic step, a decision on grouping of subjects 
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by the investigator, and a Kruskal scaling step. Table IX 
presents the percentage variance accounted for by the first 
nine factors for each of the eight data sets taken 
separately. Since the last six data sets were all scaled to 
mean 10.0 and standard deviation 2.0, the first factor in 
these cases is a mean factor. In these cases, the first 
factor was not included in the varimax rotation to simple 
structure. All subject loadings on this factor were, within 
rounding error, equal. In the first two data sets the first 
factor is probably also best interpreted as a mean factor. 
However, since the percentage variance is substantially 
lower in this case, and since subject loadings were not 
equal on the first factor, the factor was retained for the 
first two data sets. Percentage variance for the remaining 
factors in the last six cases is expressed as a percentage 
of variance remaining excluding the first factor. This 
accounts for the fact that the percentage variance accounted 
for in the table exceeds 100 percent. For the unscaled data, 
the first two sets, the first factor was retained for 
rotation. Scree tests perfomed on the percentage variance 
suggest in some cases {e.g., data set 8) how many factors 
should be retained, but in most cases the decision is not 
clearcut. Since the object of the rotation is to form groups 
of subjects, it was arbitrarily decided that four groups 
would be a reasonable number to attempt to form on the basis 
of each data set. For this reason, four factors were 


retained for varimax rotation. Por the first two data sets, 
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TABLE IX 


UNROTATED TUCKER FACTORING-PERCENTAGE TOTAL VARIANCE 
ACCOUNTED FOR BY FIRST NINE FACTORS 
FOR EACH DATA SFT SEPARATELY 
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factor. 
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factors one to four were used, while for the remaining data 
sets, factors two to five were used. Loadings of the 
subjects (Tucker's V - loading of people on principal 
vectors) on the unrotated factors are presented in Tables 
XXIII to XXX in Appendix B. 

The loadings of Tables XXIII to XXX were rotated to 
Simple structure by the varimax rotation. Results are 
reported in Tables XXXI to XXXVIII in Appendix B. An 
objective method of deciding group membership on the basis 
of these loadings was devised according to the following 
scheme. If a subject loaded greater than 1.00 on more than 
one factor, he was not placed in a group. If a subject's 
highest loading was less than one and one-half times as 
large as his second highest loading, he was not placed in a 
group. If a subject loaded less than 0.50 on all four 
factors, he was not placed ina group. Subject to these 
rules, subjects were grouped according to their highest 
loading. By this method, mutually exclusive but not 
exhaustive groups were formed. Group membership is presented 
in Table X. Overall, 61 percent of subjects were assigned to 
groups. It is probable that those excluded from groups 
produced data which differed substantially from the group 
average. Such subjects may have contributed a relatively 
large proportion of what was labelled error in the data. As 
could be predicted from the size of the goodness-of-fit 
measures between instruments, there are no consistencies of 


group membership across the four groupings at each grade 


nef ' a _¢ a Se 


Pans 


A+eh we kadeoes oft TOT elitw yhecw “—y wy of tne ctor 
dt te apmkbeo’ head “abW evit of owt ae ee he 
teafoatas no oefqesa 16 paibeol - ¥ #8 texont) atetdya 


nefdwr ot bodneasig 876 eioios? betetomnm edF% a0 {etotoev 
8 xibmaaqh mi ¥¥8 Of TIRE 

ae hatetnt etew XL8¥ OF PrIsY solde?T to spathsofk off 
ert dtowse woiietoy x*eeigne sat yd oaudoudte eiquia 
aco gibaooas ob EREURRE of) Txxn cofde® yak) oRamaayes 
peed off a tedecedanm Gaesp paifioeh to bodsea evisootdo 
sivotle? oft of prokhtooos heeivoh ase epnthsol eaedt 72 
ape sna no 00.? medt tersetP hehwol toatdua 6 21 oasdna 
aleoatdue « TE .-@RO7P e nt bestia fon aby off rote — 
i aa sfod-eno finn @a0 ned+ eeok 2e¥ paibsol seodpid 
i¢ honed) Jon pew aa ypaibsol teodpid hoones std 26 eapisl 
toot fie ao 0@e0 nele enol fabsol toetdue 6 II .quoip 
ods of tpetdue .odomp 6 OE heortq ‘on BBW od 2703982 
api <fed* 0% pakivosgs Beavorp 7°" atootdue + eotwa 
ren due ovéaufoxs _iteosum ybodtor atds q2 entbsot 
hntnapete ef aide tednem quot? .fomzo? or9ew 2aqNotD ove sou nixo 
ny hodptens stew eioefdee Be fnonied fa ,[istevo 2% oidet al 


avegp soTt hefgotozes nods endt 3 oldedotd al n — 
ssoTy oft mond y Lieb tnetnton ferotiibh doidwv stab tooubore 
olayitelea & hetudistnop SeFet~ Yon atoofdwe dove 
of antoh ode AL 10379 tel tedet aev thdw to notsrono39 
(j7-30=-sRonboay eit To ate edt sort betoiber ® : 
() embonedetaacs on ers oTedy,  etaonvr tend naseted — 


shetp iso 46 epniguete WOR edt aboT3s, 


135 


TABLE X 


SUMMARY OF GROUP MEMBERSHIP 
BY TUCKER SCALING 
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level. Such groupings of subjects together based upon 
Similarity of responses within more than one test would have 
offered strong evidence for convergent validity of the 
instruments. 

Group average similarity matrices were formed for 
each of the subgroups reported in Table X. Kruskal scaling 
of the 32 average matrices was performed. Stress values for 
these Kruskal solutions for from six through two dimensions 
are reported in Table XI. The increase in stress values from 
the large group average scaling (see Table Vv) to the 
subgroup scaling must be attributed to a corresponding 
increase in error in the data. This is despite the fact 
that, on the average, only 61 percent of the subjects passed 
the requirements for group membership based upon the Tucker 
scaling results. Even though, presumably, the most stable 61 
percent of the data were retained, the smaller size of the 
subgroups compared to the full groups must have increased 
the sampling error of the average similarity ratings. A 
larger group whose similarities were being averaged would 
allow the errors of measurement in individual similarities 
to "smooth out", The patterns of stress values in Table XxI 
may be interpreted on the basis of Wagenaar and Padmos! 
results (1971) as indicating a high percentage of error in 
the data. While some of the stress values are somewhat 
higher or lower than others for the same number. of 
dimensions {e.g., constrained association unscaled grade 
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TABLE XI 
STRESS VALUES FOR KRUSKAL SCALING 
OF TUCKER SUBGROUP AVERAGE MATRICES 
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differential grade nine group 2, three and four dimensions), 
there seem to be no substantial reasons for reporting four 
dimensional solutions instead of three dimensional 
solutions. Therefore, the three dimensional Kruskal scaling 
results for the 32 subgroups are reported in Table XXXIX in 
Appendix B. These solutions have been rotated to the best 
fit with their respective group averages. 

These Kruskal scaling results were rotated by the 
nonmetric orthogonal Procrustes rotation to each other in 
all possible combinations of the large group average and the 
four subgroup averages within each of the eight data sets. 
The resulting goodness-of-fit measures are presented in 
Table XII. In this and similar subsequent tables, "group 0" 
refers to the full group of grade nine (N=78) or grade 
twelve (N=54) subjects. From the goodness-of-fit measures it 
can be seen that in general the Tucker scaling method has 
identified different patterns of responses to the concept 
relationship tasks. The subgroup patterns tend to bear 
little resemblance to each other, and almost no resemblance 
to the total group average results. Note that the value fr 
based upon SS, the symmetric coefficient of alienation, 
cannot be negative. Detailed discussion of the differences 
in perceptual patterns will not be entered into until the 
personality variables of the subgroup members are discussed 
in the next section. Analysis of the structural patterns of 
the 32 subgroups without reference to the personality 


variables would not address the hypotheses of this 
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TABLE XII 


GOODNESS~-OF-FIT AMONG TUCKER SUBGROUP SOLUTIONS AND WITH 
RESPECTIVE FULL GROUP SOLUTIONS 
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Grade 9 Groups Grade 12 Groups 
0 1 2 3 u 0 71 2 3 4 

CAU 

Grp 90 18 a3 14 15 28 23 28 17 
2 98 ay 49 51 96 33 31 37 
2 99 90 ST 4 96 94 38 40 
3 99 87 82 57 96 95 92 45 
4 99 86 85 B82 93 93 92 90 

CAS 

Grp 0 12 22 22 12 25 18 ae 26 
1 99 44 31 48 97 39 56 Fie) 
2 98 91 35 4 98 92 49 49 
3 98 95 93 “4 99 83 86 ah) 
4 99 85 eo 91 96 87 87 BY 

SJ 

Grp 0 26 10 1a nae 19 17 10 16 
1 97 42 36 a5 98 38 58 62 
Z 99 91 a7 61 99 92 29 34 
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SD 

Grp 90 23 32 23 38 ne 712 19 29 
1 o7 56 33 46 98 a7 42 25 
2 95 83 45 oN 99 93 57 53 
3 o7 a5 89 36 98 91 80 46 
4 92 89 86 93 96 94 85 89 


Below diagonal - SS; above diagonal - r 
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investigation. At this point in the discussion, it is 
possible to conclude that the technique of Tucker subgroup 
formation has isolated subgroups whose Kruskal 
configurations have very poor fits with each other and with 
the full group configuration. generator oe from these data 
to individual differences in cognitive structure as 
hypothesized will be left until the discussion of the 


personality variables of the subgroups which follows next. 


4.6 Tucker Scaling Results-Personality Variables 

The proper procedure to follow in searching among 
the subgroup members for differences in the personality 
variables would be a multivariate analysis of variance 
procedure. This procedure was attempted, but the inverses of 
the matrices of personality variables could not be 
calculated because of high correlations among the 
personality variables. That is, the determinant of the 
Matrix approached zero. The problem can be avoided by 
deletion of one of the redundant variables. Since in the 
present data the correlations under consideration were not 
exactly equal to 1.00, it would make a difference which 
Variable were discarded. Rather than risk deletion of an 
important variable, one way analysis of variance was chosen 
as the analytic technique. Because of the danger of finding 
a significant difference by chance alone, all tests of 
significance were at the 0.01 level. A summary of analysis 


of variance results is reported in Table XIII. The first 
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TABLE XIII 
ANALYSIS OF VARIANCE-PERSONALITY 
VARIABLES AMONG TUCKER SUBGROUPS 


—— em ee ee ee ee ee ee ——_ 
—_ — ee ee ee ee a a a a ee a eee eee 


CAU CAS SJ SD 

2 p iY p ig Pp F Pp 
Grade 9 
Age Yeo 268, Tot Rae 0.6 .60 ae es 
Sex Ont. wits ienee ue ee 056 65 
To Qs Gat «534 G.1 2.394 te ee pe 
Achievement Ret 2/8 Ae, 29 26628 0.81253 
Hidden Figures iow 65.16. 023 236 0.4 .74 er Le 
Developmental Level 1.865214 05..62 D4 =7F ge ee a 
Cognitive Complexity Ved) «83 Zao Terese 0.4 .78 
Consistency-SJ Date 2 99168. 286 0.4 «OS 05305 
Consistency-SD arp? meses % em, ees | apa! 0.8. 299 CR nS 
Consistency-Ca jet eal” ton sae 0.9 44 teh. 26 
Total responses-CA I02 ~002 10.39 0.3 .34 0.47 94 
Internal responses-CA 6.7 .001 4.4 .010 1.8 .16 > hae st be 
Fraction internal-CA Bed) at OO Sale) aa ele tase 5 Is HOE hers 
Avg.relatedness index 16.6 <.001 5.2 .004 1.1 .36 1.00641 
Personal Consistency pe Ser er ee Ar eae | 0.6 «04 es ae Be 
Grade 12 
Age ed eee eset ne pA 138 220 
Sex Dern. Peer eos Ooo 5 yi eee 
Achievement tat) “2 0G0 ULecute O58) oS Te ore 
Hidden Figures PR re LE ee ee BS Ce ee 052-538 
Developmental Level 94 «Ft ineh S27 Ze) ous 0.4 174 
Cognitive Complexity 22. eS ee eee tas ERG Sage dae 
Consistency-SJ Vee = So - Ustaasr> Ze aed ce: oy ee 
Consistency-SD 2e5> «02 @OC2 8S 0.9 .48 OSes 22 
Consistency-CAa [re 4 ae ee as ot Pe eas 0.3 «82 
Total responses ee er bm ts eee 0.8 .49 Aa she hd 
Internal responses-CA 18.2 <.001 2.0 .13 0.6 .64 Pp a: 
Fraction internal-CA 10.4 <.001 2.1 .12 05.8 «54 Os2 5916 
Avg.~relatedness index 12.9 <.0C1 2.1 .12 0.6 ~62 Ue Pee ee 
Personal Consistency Ve Te 6 e59Bo 263)-208 0.9 543 0.2. 6.83 
Notes: 


1. Three decimals reported for significant or 
near significant differences only. 

2. df - groups = 3 for all data sets 
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four variables in the table are self explanatory. The next 
three were discussed in section 2.5. The four consistency 
measures were discussed in sections 3.2 and 3.3. The 
remaining variables pertain to fluency on the constrained 
associaticn test. "Total responses" is the total number of 
responses produced by the subject. "Internal responses" is 
the total number of responses consisting of words from the 
original list of 15 stimulus concepts. "Praction internal" 
is “internal responses" divided by "total responses", 
Average relatedness index measures the degree of 
cohesiveness of the pool from which responses are drawn. The 
finding of significant differences in personality variables 
among subgroups would indicate that the subgroups who have 
already given evidence of differences in cognitive structure 
also differ on personality variables. It was a hypothesis 
that differences in cognitive structure could be related to 
differences in personality variables. The only significant 
differences found were on those variables generated 
internally by the constrained association test among the 
groups formed from the unscaled constrained association 
data. Such a difference, while interesting and not 
unexpected, is not of major theoretical importance. It is of 
interest, however, that the differences in fluency variables 
found for the unscaled constrained association data did not 
persist to the scaled data. It may be concluded at this 
point that the rescaling of relatedness indices, although it 


does not, according to the results reported in Table VIII, 
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make any appreciable difference to the full group average 
Kruskal results, does avoid the formation of subgroups based 
primarily on fluency. Whether or not such formation is 
desired is a matter for judgment in the context of the 
particular investigation under consideration. Systematic 
checking of about one-third of the 96 dimensions of the 32 
scaling solutions revealed only the occasional interpretable 
dimension. 

The lack of significant differences among the 
theoretically important personality variables and the lack 
of suitable interpretations for the scaling solutions can be 
traced to several possible reasons. First, error in the 
concept relationship data, error of measurement in the 
personality variable measures, and high stress for the 
scaling solutions may have combined to obscure any 
potentially significant results. Second, the personality 
variables may not be related to differences in cognitive 
structure. Thus, members of the subgroups may have widely 
different cognitive structures, with the resultant average 
being uninterpretable. Third, if the perceptions of the 
subjects were constrained by the "right answers", as they 
possibly might have been in the Kass (1971) study, then the 
Tucker scaling may have loaded those subjects close to the 
group average in perception to approximately the same extent 
on all dimensions. Thus, according to the criteria for 
subgroup formation such subjects would not be included in 


any of the subgroups. As a result, the Tucker process may be 
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isolating only those subjects who deviate from the group 
average substantially. The data of subjects who deviate from 
the group average a great deal may be largely random error. 

The lack of interpretability of the Kruskal 
solutions for the Tucker subgrcups suggests that the Kruskal 
technique has not functioned productively in the subgroup 
data analysis. Since the Kruskal technigue did function 
successfully on the large group data, it seems reasonable to 
conclude that, with a smaller sample size in the subgroups, 
the sampling error was relatively larger, and the percent 
error in the data became too large for the revealing of 
meaningful dimensions. If this is the case, then these 
results suggest that, in fact, the Wagenaar and Padmos 
criteria for acceptable stress may be too lenient. 

Doubt has been cast upon the operation of the Tucker 
technique for the formation of subgroups, and upon the 
Kruskal technigue on subgrcup average data. It was a 
hypothesis of this study that differences in cognitive 
structure would be connected tc differences in personality 
variables. Based upon the results reported thus far, it 
would appear that it will not be possible to address this 
hypothesis in the present study. It has not been established 
that the Tucker technique is forming subgroups on the basis 
of meaningful individual differences. If not, then the 
possibility has not keen ruled out that a "good" separation 
into subgroups would yield individual differences in 


theoretically important personality variables. Such a 
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conclusion would he in keeping with the suggestion that the 
Kruskal solutions of the Tucker subgroups are 
uninterpretable because the data of the subgroup members is 
essentially meaningless. If however, the Tucker technique is 
operating as desired, then the lack of individual 
differences in personality variables must be traced to 
inappropriate selection or unreliable measurement of the 
personality variables. The lack of interpretability of the 
Kruskal solutions of the Tucker subgroups, then, must be 
traced not to meaninglessness of the similarity data, but to 
the lack of robustness of the Kruskal technique with respect 
to sampling error in smail samples. Based upon results 
reported in section 4.8, it was decided to analyse the 
Tucker subgroup data by Carroll analysis. Such results are 
reported in section 4.8, and shed light on the hypothesis 


under discussion. 


4.7 Carroll Scaling Results-Cognitive Structures 

Carroll scaling was performed on the eight sets of 
data. In order to check for local minima, the analysis was 
performed twice, using first a random number starting 
configuration, and second the group average Kruskal results 
as starting configuration. Table XIV presents the goodness - 
of-fit relationships among the two Carroll solutions, the 
Kruskal solutions, and the data. From Table XIV, it can he 
seen that there is little to choose between the "random 


number" start and the "Kruskal" start Carroll scaling 
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RELATIONSHIPS AMONG CARROLL SOLUTIONS, 


TABLE XIV 


SOLUTIONS, AND DATA FOR FULL GROUPS 


KRUS KAL 


—_—— a nr en ee ee eae ee a ae ae i ae ea ae ae ee ee 


Data sets 1 Z 3 4 5 6 7 

CAUY 9.02 1200 056700. 79 0.62° 0.980.806 

CAU12 9298 043599 0.66 0275 0.63 0.77 (0258 

CAS9 0.092°40200' $2570.86 0.52 0.85 9650 

CAS12 0. 7% 0.99 0.67 0.78 0.65 0276 0.558 

SyE BS) 0.75 0.99 0.31 9.95 0.323 0.95 6.64 

S312 0.14 0.99 0.48 0.88 0.48 0.88 0.59 

sb9g 6200082 00002:55 70. B4-0559.0.8%4 90250 

SDd12 0203°4200°05348' 0.9% 0.34 0.9% GsS4 

1. Fit of "random start" and "Kruskal start" Carroll 
results to each other - SS 

2. Fit of "random start" and "Kruskal start" Carroll 
results to each other - r 

3. Fit of "random start” Carroll results to Kruskal 
group average results - 55 

4. Fit of “random start" Carroll results to Kruskal 
group average results - rf 

5. Fit of “Kruskal start" Carroll results to Kruskal 
group average results - SS 

6. Fit of "Kruskal start" Carroll results to Kruskal 
group average results 

7. Average correlation of "random start" results to data 

8. Average correlation of "Kruskal start" results to data 
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results. Their fits to each other are almost perfect, and 
their fits to the Kruskal group average results are 
virtually identical. Perfect fit to the Kruskal group 
average results was not expected. In the Kruskal technique, 
all subject ratings are given equal weight by the averaging 
process. In the Carroll technique, on the other hand, 
subjects closer to the group average are more heavily 
weighted. Since there was no criterion by which to choose 
between the two Carroll analyses, the "random number" start 
results were arbitrarily chosen for subgroup formation and 
further reporting. 

Unlike the Tucker scaling technique, the number of 
dimensions on which subjects are separated and the number of 
dimensions on which concepts are rated are the same. In the 
Tucker analysis, it was possible to use four dimensions of 
the principal components analysis in order to form subgroups 
and still report three dimensional cognitive structures for 
those subgroups. In the Carroll technique, subject space and 
Stimulus space must be of the same dimensionality. The 
Carroll analysis was carried out in four dimensions as well 
as in three. Over all eight data sets, the average 
correlation between the data and the Carroll solutions was 
0.62 for four dimensions compared with 0.58 for three 
dimensions. The slight increase in this value, which is 
analogous to a goodness-of-fit measure, was not considered 
worth the sacrifice of interpretability which the addition 


of a fourth dimension would entail. The three dimensional 
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solutions are reported, and were used to form subgroups. 

The Carroll results can produce subgroup cognitive 
structures directly by calculation of weighted composites of 
the loadings of concepts on each dimension. Saliences of 
dimensions for subjects are the weights used in this 
process. This was the procedure by which the elegant 
solutions of the "rating of nations" or the "colourblind" 
examples were obtained. In the present situation, the 
average correlation of the results with the data is 
relatively low. Rather than opt for the weighted composite 
procedure, it was decided to use the weights of subjects on 
dimensions to produce subgroups ina manner analogous to 
that used with the Tucker results. That is, subjects were 
grouped according to their saliences on dimensions, but 
their raw data were then used for averaging to produce a 
subgroup Similarity matrix for Kruskal scaling. As noted 
above (P.48), this procedure is one less best fit step 
removed from the raw data than the procedure recommended by 
Carroll. 

The criteria introduced for subgroup membership 
were, of necessity, different from those employed with the 
Tucker results. In the Tucker scaling technique, the 
dimensions produced in the principal components analysis 
appeared to bear no direct relationship to the dimensions 
along which subjects rated concepts. Loadings on dimensions 
indicate only similarities and differences in the raw 


similarity judgments. Thus, a subject who loaded highly on, 
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for example, both dimensions two and three could justifiably 
be considered not to be in the same subgroup as those who 
loaded highly only on dimension two or only on dimension 
three. On the other hand, the Carroll analysis produces 
subject loadings which indicate directly the importance of 
each of the underlying dimensions of judgment to the 
subject. If we are attempting to isolate those subjects for 
whom, for example, dimension two was important in their 
ratings of concepts, then a high loading on another 
dimension is not sufficient grounds for excluding a subject 
From the "dimension two" group. The criteria for subgroup 
formation based upon the Carroll analyses were: for each 
dimension, take the ten highest (grade nine, N=78) or eight 
highest (grade twelve, N=54) loadings on each of the three 
dimensions as indicating group membership. Such a procedure 
appears reasonable because it groups 41 percent of subjects 
in three groups compared to 61 percent in four groups for 
the Tucker grouping. Since, therefore, the average subgroup 
is approximately the same size in both the Tucker and 
Carroll groupings, homogeneity and sampling error should he 
comparable. Three groups were formed from each data set, 
giving 24 groups in total. Unlike the Tucker Taine: these 
groups were not mutually exclusive. Subject loadings on 
dimensions for the Carroll analyses are presented in Tables 
XL to XLVII in Appendix B. Group membership is presented in 
Table XV. 


Substantial commonality of group membership was 
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found. With the exception of two pairs of subjects, grouping 
in the constrained association test groups for the scaled 
and unscaled data are identical for both grades. For grade 
nine, constrained association unscaled group one and 
Similarity judgment group three have six members in common. 
Group two of the similarity judgment grade nine has four 
members in common with each of the first two groups of the 
constrained association data. Also for grade nine, semantic 
differential group one has four members in common with 
Similarity judgment group three. In the grade twelve 
memberships, constrained association unscaled group one has 
four members in common with similarity judgment group one, 
and constrained association-scaled group three has three 
members in common with similarity judgment group two. These 
commonalities are in marked contrast to the groupings based 
upon the Tucker scaling results. While the grouping criteria 
in the Tucker technique prevented the subject from being 
placed in more than one group within a data set, there is no 
such restriction applied to the subjects with whom he is 
grouped across data sets. The commonalities between the 
scaled and unscaled versions of the constrained association 
data are predictable on the basis of the strong fit between 
the Kruskal group average results for these four data sets. 
These commonalities, and even more so the commonalities 
across different similarity rating instruments are strong 
evidence both for the construct validity of the cognitive 


structures uncovered, and for the power of the Carroll 
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scaling technique. The fact that, responding to different 
Similarity instruments, subjects were grouped together in 
the Same groups on the basis of response similarity 
indicates a high consistency of response across instruments. 
This is deemed to be evidence for the construct validity of 
cognitive structure. 

The subgroup average matrices were analysed by the 
Kruskal technique in order to seek evidence for differences 
among subgroups in cognitive structure. Stress values for 
five, four, and three dimensions are presented in Table XVI. 
As with the Tucker results, the stress values are quite 
high. Again, this may be traced to the small group size. 
There are no trends in stress evident either within Table 
XVI, or from comparing this table with the corresponding 
table (Table XI) for the Tucker groups. Kruskal solutions 
for each of the 24 subgroups were rotated within each data 
set by the nonmetric orthogonal Procrustes rotation. 
Measures of fit are summarized in Table XVII. A comparison 
of this table with the corresponding table for the Tucker 
group rotations (Table XII) shows measures of fit of the 
same order of magnitude for both subgroup results. The 
Carroll subgroup results fit each other and the full group 
results slightly better than the Tucker subgroup results. 
The average correlations with the whole group average and 
among the subgroups are, for the Tucker results, 0.20 and 
0.44 respectively, while the corresponding values for the 


Carroll results are 0.24 and 0.48. The Kruskal scaling 
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TABLE XVI 


STRESS 1 VALUES FOR KRUSKAL SOLUTIONS OF 
CARROLL SUBGROUP AVERAGE MATRICES 
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TAOLE £Vit 
GOODNESS-OF-FIT AMONG CARROLL SUBGROUP 


KRUSKAL SOLUTIGNS AND WITH RESPECTIVE 
FULL GROUP KRUSKAL SOLUTIONS 


_— Oe ee ee ee ee ee ee ee ee ee eee eee ae a a eee eee ee i ee ee 


Grade 9 Grade 12 
0 1 2 3 Q 1 2 3 
CAU Grp 0 a3 17 18 ba 33 25 
1 97 43 50 97 yy 44 
2 98 90 ug 95 88 36 
3 98 87 87 97 91 93 
CAS Grp 0 27 25 15 19 30 20 
4 96 52 56 98 50 61 
2 97 86 43 95 87 50 
3 99 83 90 98 79 8H 
SJ Grp 0 26 20 16 25 42 18 
1 96 Q7 35 97 47 35 
2 98 88 63 99 88 ae 
3 99 94 78 98 94 64 
SD Grp 0 29 24 22 25 38 36 
1 96 51 a 97 a4 46 
2 97 86 49g 92 95 45 
3 98 94 37 93 89 90 


eee ee net ee ae ee ee ee ee ee ee ee ee ee ee ees ee ne ee eas ee eee ene er een nee nena ee 


Below diagonal - SS; above diagonal - r 
Decimals omitted _ 


i | 


TEVX 258AT 


sj 


crotiuine JdORMA HAOnA T19-10-22 740008 


2YTPOITAIN HTEE THA cv 


OITUIOR LASTER 


sxormiree pameues gvoda JI0T 


t Ff 
tr On £ 
tf 2 Kr 
tra | WS Ti he 
vf AA ct ea 
$f tr le 
vo 
os vf ro | 
i) fe qe a? 
x + a ep te 
TAY Re 
wt at 
; re cf 
ry f ap ta 
it iP 
fe mw ?@ cc 
ip t' Ey fu 
“ "4 qo Py 
i a 


¢ ofete 
g r Q 
vr ES 0 @29 UAD 
fe ve r 
oe AP ¢ 
Ts 8 ae E 
es LS 0 q1d 2@AD 
Se ae r 
oF ve S 
oe ff ee 2 
OS as 0 qzo Le 
To ae t 
PA 8e g 
at ve, ee E 
as es Oo am Ge 
te av r 
a8 Fe S 
Ts re Be ft 


‘ 
aan ae oe ee ee <r ae ee ws nt i gees en tae ein ie So St OO ES ES SS NS SE, SE SERRE ESS 


+ = InnopHib evode 


22 - Isnopeth woled 
hettimno alsaioed 


155 


solutions for the 24 subgroups are reported in Table XLVIII 
in Appendix B. Further discussion of the interpretation of 
these scaling results will be left until section 4.8, which 
will present the differences in personality variables among 


the subgroups. 


4.8 Carroll Scaling Results-Personality Variables 

As with the examination of the personality variables 
of the Tucker subgroups, this part of the analysis was 
carried out by one way analysis of variance, using a 
criterion of 0.01 for statistical significance. Because of 
the somewhat dubious appropriateness of using one way 
analysis of variance, there is a danger of Type 1 error in 
searching for differences in personality variables among the 
subgroups. In order to minimize this danger, the stringent 
criterion of 0.01 has been used. A summary of the 
Significance levels of differences in the variables is 
presented in Table XVIII. The results are similar to those 
for the Tucker groups. Significant differences in two cases 
for one of the internally generated variables were found, 
and a sex difference emerged in another two cases. Rotations 
of these subgroup solutions to a best fit with the 
appropriate Kruskal group average solution revealed a 
general lack of interpretability of the solutions. The 
Situation is similar to that involving the Tucker solutions, 
except that in the Carroll case, a measure of correlation of 


the solution with the data is available. As reported above, 
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TABLE XVIII 


ANALYSIS OF VARIANCE-PERSONALITY 
VARIABLES AMONG CARRCLL SUBGROUPS 
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Notes 
1. Three decimals reported for significant or 
near significant differences only. 
2. df - groups = 2 for all data sets 
3. df — error = 27 for grade 9 and ZT for grade V2 
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the average correlation of the Carroll solutions for all 
subjects over all data sets was 0.58. However, for those 
subjects who met the criteria for subgroup membership, this 
average correlation was 0.68. This substantial difference 
prompted the investigator to perform Carroll scaling on the 
subgroups in order to achieve solutions which would not 
require rotation to meaningfulness. Similar analysis, as 
mentioned above, was performed on the Tucker subgroups. The 
Carroll solutions of the Carroll subgroups are reported in 
Table xXLIX, and those for the Tucker subgroups in Table L, 
both in Appendix B. The interpretations of the dimensions 
uncovered are summarized in Table XIX for the Carroll 
subgroups, and in Table XX for the Tucker subgroups. As can 
be seen from Table XIX, 80 percent of the dimensions for the 
Carroll subgroups can be assigned meaning on the basis of 
the criteria used in the analysis of the large group 
solutions. As can be seen from Table XX, about 70 percent of 
the dimensions are interpretable for the Tucker subgroups. 
Substantial individual differences among the solutions can 
be seen. The fits of these Carroll solutions to the 
corresponding subgroup average Kruskal solutions are also 
included in Tables XIX and XX. Since it was not possible to 
identify the members of the subgroups, little will be 
achieved by a detailed discussion in terms of the concept 
meanings of the manner in which the cognitive structures 
differed. However, the meaningfulness of the solutions 


suggests that, in fact, the Tucker scaling technique is not 
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TABLE XIX 


INTERPRETATIONS OF DIMENSIONS IN CARROLL SOLUTIONS 
CF TUCKER SUBGROUPS AND FIT WITH KRUSKAL SOLUTIONS 
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TABLE XX 


INTERPRETATIONS OF DIMENSIONS IN CARROLL SOLUTIONS OF 
CARROLL SUBGROUPS AND FIT WITH KRUSKAL SOLUTIONS 
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operating solely on chance fluctuations, but on some real 
but unidentified individual differences. 

It should be noted that the assignment of labels to 
dimensions is a very subjective process on the part of the 
investigator. The dimension labels used have enough 
conceptual overlap to render almost impossible the 
replication by another judge of the labels assigned. Because 
of this difficulty, it has been decided not to proceed with 
attempts to link differences in cognitive structure with 
differences in personality variables, for the few cases 
where such differences were found. Such linking would likely 
amount to overinterpretation of the data. 

The goodness-of-fit measures among the Carroll 
solutions for full groups and Carroll subgroups are reported 
in Table XXI, and among the full groups and Tucker subgroups 
in Table ra Ae Re The goodness-of-fit measures are 
substantially higher, both among the subgroups and with the 
full groups, than the corresponding measures for the Kruskal 
solutions (see Tables XII and XVII). The magnitudes of the 
values in Tables X¥XI and XXII indicate a reasonable balance 
between no individual differences (as would be indicated by 
SS values of 0.00) and individual differences so extreme (as 
would be evidenced by Tables XII and XVII, assuming that the 
Kruskal subgroup solutions had been clearly interpretable) 
that little commonality of perception among subgroups could 
be imputed. For the purposes of addressing the hypotheses of 


this study, even though the dimension interpretations 
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TABLE XXI 


GOODNESS-OF-FIT MEASURES AMONG THE CARROLL SOLUTIONS 
OF FULL GROUPS AND CARROLL SUBGROUPS 


—— ee re ee ee nce cr se ne ee cee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ae eee ee ee 


Data Grade 9 Groups Grade 12 Groups 
Sets 
9 1 2 3 0 1 2 3 

CAU Grp 0 90 80 79 88 86 1 
1 45 69 66 48 78 70 
2 60 v2 75 50 62 55 
3 62 75 66 66 72 84 

CAS Grp 0 90 89 80 86 80 IS 
1 43 80 7 AZ 93 70 
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z} 60 68 72 66 a1 the 
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3 35 65 55 68 Ti 790 
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3 43 a7 a3 62 716 81 
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TABLE XXII 


GOODNESS-OF-FIT MEASURES AMONG THE CARROLL SOLUTIONS 
OF FULL GROUPS AND TUCKER SUBGROUPS 


_—— ee ne ee ee ee ee ee ee ee ee ee ae ae ee eae ee ee ee i 


Data sets Grade 9 Groups Grade 12 Groups 
0 1 Zz 3 a 0) 1 2 3 4 
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themselves are no longer of interest without identifiable 
differences among subgroup members, the fact that such 
labelling is possible, and the fact that individual 
differences in configuration were found as evidence of 
individual differences in cognitive structure are relevant. 
A major hypothesis of the study was that individual 
differences could be found in cognitive structures relating 
the 15 concepts under investigation. Such differences were 
found by both the Tucker and Carroll techniques of subgroup 
formation. The subgroups formed by the two techniques, 
however, bore no relationship to each other. The personality 
variables used for subject identification in the study were 


not useful as descriptors of subgroup members. 


4.9 Summary 

The hypotheses of this study sought evidence for 
convergent validity of cognitive structures through analysis 
of Similarity data obtained through three different 
instruments from the same subjects, and indirect evidence 
for discriminant validity of cognitive structures through 
identification by independent means of subgroups which 
differed in cognitive structure. Convergent validity was to 
be enhanced by applying two scaling techniques to the data. 
These techniques differed in assumptions and computational 
methods. Discriminant validity was to have been enhanced by 
explanation of differences in cognitive structure by 


differences in independent personality variables. 
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Based upon the group average data {see 4H0(1), 
section 1.7), convergent validity between the constrained 
association data and the similarity judgment data was judged 
satisfactory. The fit of the semantic differential results 
to the other two was considerably lower. Using the grade of 
subjects as an independent variable, it was not possible to 
produce the low goodness-of-fit measures between grades 
which would have been positive evidence of good discriminant 
validity. Since it is not clear on theoretical grounds that 
grade should bea discriminating variable, it is difficult 
to assess the importance of this lack of discriminant 
validity. | 

For 4H0 (2) and H1(2) (section 1.7), the measures of 
two of the personality variables, the developmental level 
test and the Hidden Figures Test were judged to be 
sufficiently reliable for their intended purposes of subject 
identification. The cognitive complexity test did not prove 
to have sufficient test-retest reliability. None of the 
independent variables chosen for study had any value in 
identifying subgroup members. Thus, it was not possible to 
link differences in cognitive structure with differences in 
personality variables. 

The Tucker and Carroll techniques both appeared to 
operate successfully in separating the subjects into groups 
with differing cognitive structures. These separations, 
however, bore no relationship to each other. Since no 


consistencies of group membership were found between the 
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Tucker subgroups and the Carroll subgroups, the approach to 
convergent validity which used similar results based upon 
two different analytic techniques proved fruitless. There 
was evidence that the error level of subgroup data was too 
high for the Kruskal technique to uncover meaningful 
cognitive structures. If this is the case, then Wagenaar and 
Padmos (1971) may have underestimated the effect of error on 
stress for the Kruskal technique. According to their 
criteria, the stress of the solutions of the subgroups was 
not high enough to indicate that the proportion error in the 
data would render interpretation meaningless. 

Using the Carroll technique on the subgroup data, 
meaningful and different cognitive structures were recovered 
for the subgroups. In finding meaningful individual 
differences in cognitive structure, the study has achieved 
some degree of success. Dimensions labelled "certainty", 
"creativity", "temporal", and "theoretical-practical" were 
identified in various combinations as being reasonable 
interpretations of subject scaling result dimensions. In 
linking these differences to independent variables, the 
study has not been successful. The next chapter will discuss 
the educational implications of this line of research, will 
suggest possible improvements to this study, and will make 


recommendations for further research. 
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CHAPTER 5 


SUMMARY, IMPLICATIONS, AND RECOMMENDATIONS 


5-1 Introduction 

This chapter will proceed in three main sections. 
The next section will summarize the study. Section 5.3 will 
discuss the educational implications of the study. Section 
5.4 will suggest how this study could have been improved 
upon while maintaining essentially the same format, and will 
discuss further suggested research which may add to the body 


of knowledge concerning cognitive structure. 


5.2 Summary of Study 

This study examined the construct validity of 
cognitive structures, defined as sets of relationships among 
concepts. Construct validity was assessed primarily by 
convergent validity (Campbell and Fiske, 1959) and 
indirectly by discriminant validity. The investigation 
proceeded by multidimensional scaling analysis of concept 
Similarity data obtained from 78 grade nine science and 54 
grade 12 biology students. Using a set of 15 syntactical 
concepts from the domain of the scientific method subjects 
responded to three concept rating tests: a constrained 


association test (Garskof and Houston, 1963), a similarity 
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judgment test (Torgerson, 1958), and a semantic differential 
test (Osgood, 1957). Personality data were also gathered 
from the subjects in the form of tests of developmental 
level (Inhelder and Piaget, 1958: ‘Hobbs; *TS75yfy;7° “field 
indepen dence-dependence (French et al, 1965), and cognitive 
complexity (Seaman and Koenig, 1974). 

Validity was investigated through the scaling 
technique of Kruskal (1964a; 1964b) applied to group average 
data, and through subgroup formation based upon individual 
differences in similarity judgments as revealed by Tucker 
and Messick (1963) points-of-view analysis and Carroll and 
Chang (1970) individual differences analysis. The measure of 
convergence of scaling solutions was a non-metric version of 
the orthogonal Procrustes rotation (Schonemann and Carroll, 
1970; Lingoes and Schonemann, 1974). 

It was hypothesized that individual differences in 
cognitive structures would be linked to individual 
differences in personality variables. In addition to the 
three tests of personality variables administered, other 
variables investigated included age, Sex, grade, 
achievement, TeGe;z and several variables relating to 
personal consistency of response on the similarity measures 
and fluency of response on the word association test. 

All configurations reported are in three dimensions. 
The goodness-of-fit, taken as evidence of convergent 
validity, was quite high between the group average data of 


both grades for the word association test and the similarity 
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judgment test {average r=0.90). Goodness-of-fit was 
considerably lower {average r=0.67) a the semantic 
differential data. Both the Tucker and Messick technique and 
the Carroll and Chang technique produced subgroups with 
interpretable configurations, indicating evidence for the 
construct validity of cogntive structure. Measures of 
goodness-of-fit between these subgroup solutions were 
neither high nor low (average r=0.67), indicating some 
evidence for individual differences in cognitive structures. 
Subgroups formed by the two scaling techniques bore no 
relationship to each other, but subgroups formed on 
different similarity measures using the Carroll and Chang 
method bore strong resemblances to each other. 
Interpretability of subgroup configurations produced by the 
Kruskal technique was low, possibly due to high error in the 
data. The Carroll and Chang technique, however, produced 
interpretable results, and thus was judged relatively robust 
with respect to error. 

The study provided some construct validity for the 
interpretation of configurations as cognitive structures 
based upon data obtained by word association and similarity 
judgment. Individual differences in cognitive structure were 
found in the configurations of the subgroups formed. 

None of the personality variables examined were 
fruitful in identifying members of subgroups formed. With 
the exception of the cognitive complexity test, 


reliabilities of the personality variables were 
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satisfactory, with estimates by various methods ranging from 
0.50 to 0.76. There was some consistency of subgroup 
membership within the subgroups formed by the Carroll and 
Chang analysis, but not among those formed by the Tucker and 


Messick analysis on the different data sets. 


5.3 Educational Implications 

It has been suggested (Shavelson, 1974) that, over 
the course of instruction, the cognitive structures of 
students should be altered in the direction of that of the 
instructor. While a similarity judgment test cannot be 
imagined as a useful posttest grading device, it has 
potential application as a pretesting device. The particular 
set of concepts chosen for this study have limited 
educational import. It is of interest, however, that 
scientific method is perceived by subjects as varying along 
at least three, and possibly four dimensions. Tentative 
labels for some of the subject perception dimensions were 
certainty, creativity, temporal, and theoretical-practical. 
It is possible that those subjects who perceived a 
creativity dimension to the scientific method may be nore 
aware of science as the product of human imagination’ than 
those who did not. The "theoretical-practical” dimension may 
be related to subjects’ perception of the difference between 
science and technology, a difference which is not always 
clear in the minds of the public. Similarly, the perception 


of a “certainty” dimension, while almost universal among the 
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subgroups, could be related to the view of science as an 
explainer rather than a producer of goods. This distinction 
is closely related to the science-technology distinction. 

However, for a set of more substantive science 
concepts, it may be possible to ascertain along which 
dimensions of cognitive structure a subject waS Similar to 
the instructor (or to the "target"), and along which he was 
different. Such a finding could have implications for the 
design of individualized instruction. Hunt (1971) has 
suggested that it may be possible to design instruction to 
fit a student's cognitive style, or preferred way of 
learning. It may also be possible to match student and 
teachers who have the same perceptions of a concept domain, 
or who have compatible styles of thought. Hunt also suggests 
that perhaps we should be "mismatching" students with 
teachers in order to introduce students to different 
thinking patterns and increase their cognitive flexibility. 
In either case, cognitive structure may be an educationally 
important variable for the design of individualized 
instruction. 

By extension of the technique, it may be possible to 
develop posttesting devices. Consider a situation where the 
instructors have identified the dimensions along which they 
wish subjects to be able to distinguish among a set of 
concepts. A possible posttest could be devised which 
involved rank ordering concepts along several continua. It 


is difficult to see any immediate advantages of such a 
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testing method over more traditional testing methods, but 
the notion should not be ignored because its direct 
application cannot be foreseen at present. Attempts have 
begun (Shavelson and Geeslin, 1975) to apply the techniques 
of scaling theory and graphing theory to the recovery of 
structural information from textual material. Undoubtedly, 
the same is possible for interactional analysis data. The 
technigue has the potential to become an important analytic 
tool for the analysis of nuances of meaning conveyed by 
teachers and students in classroom discourse. As noted by 
Hartnett and Rumery (1973), the mathematical analysis of 
classroom interaction data is in its infancy. 

Perhaps the most fruitful applications of the 
technique will be noticed after the high error levels in the 
data have been reduced. It may be possible to train subjects 
in a relatively short period of time to be more consistent 
in their reponses. Since this study has touched upon only 
three of the many possible methods of gathering similarity 
data, perhaps another method of data collection will prove 
more error-free. 

Assuming that the error in the data can be reduced 
substantially, further research in the area will reveal 
further possibilities for the applicatioh of cognitive 
structure. In theory, all the educational advantages of the 
precise measurement of individual differences apply to the 
recovery of structural information from student data. There 


is considerable error in the data obtained from subjects in 
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this study. It is not clear whether this error is due to 
limitations of the techniques or to lack of stability in 
perception or lack of accurate introspective ability on the 
part of the subjects. The conversion of word association 
lists to relatedness indices loses a great deal of 
information about subjects’ perceptions. The failure of many 
subjects to confine themselves to the constraints of the 
word association test was surely a confounding factor in the 
analysis. Further modification of the directions given to 
the subjects may lessen these problems. No attempt was made 
in the study to use redundant information supplied by the 
Similarity judgment test responses to estimate subject 
response stability. How this should be done is not clear, 
but redundancy of information should not be ignored in 
attempts to improve the accuracy of the similarity matrices 
produced from subject responses. The semantic differential 
test did not correlate as well as might be hoped with the 
other two similarity rating tests. Analysis of the data of 
this test by more traditional semantic differential 
techniques might shed light on the usefulness of the 
particular bipolar scales chosen for the concepts and 
subject ages studied. This was a technique of instrument 
refinement not used in the study. 

There is considerable variation in the literature in 
the nature of the directions given for the constrained 
association test. A balance must be struck between focusing 


subjects on the concept domain of interest and stifling 
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their fluency of response. For example, it is possible that 
the legal connotations of "law" and "evidence" may have 
interfered with subject responses to these concepts in the 
scientific context of this investigation. Instructions which 
ask subjects to list words they would use in explaining or 
defining the concepts, despite the predicted effect on 
Fluency of response, may prove to be more useful for testing 
in an educational context. 

The personality variables measured in this study did 
not prove fruitful in the analysis of individual 
differences. The developmental level test concentrated on 
the ability to combine and control variables at the expense 
of other aspects of formal thought, such as proportional 
ability. Because of this limitation, and because of the 
conceptual difficulty involved in estimating the reliability 
of the measure, the measure in retrospect seems inadequate 
for the task. The cognitive complexity test was even less 
reliable than previous instruments reported in the 
literature which have attempted to measure this variable. I+ 
is not clear how reliability of the instrument could be 
improved within the time constraints of this study. Assuming 
the construct is educationally useful, which is not 
established, then more testing time should have been devoted 
to its measurement. Since it would have been next to 
impossible to ask for a fourth class period, perhaps a 
better approach would have been to not administer the 


semantic differential instrument and concentrate on a longer 
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and better estimate of cognitive complexity. 

The scaling techniques used did not operate equally 
efficiently on the data provided. The Tucker technigue must 
be judged inferior to the Carroll technique because of the 
lack of consistency in group membership across data sets. 
The Kruskal technique failed to provide meaningful 
information when operating on the subgroup data, presumably 
because of the high noise ratio in the data. Because of 
this, its accuracy for the full group data must also be 
questioned. Perhaps use of the Carroll technique ina manner 
which utilized the saliences of the dimensions would have 
been more informative. The reversion to the Kruskal analytic 
technigue for the subgroup configurations Was, in 
retrospect, unwise. 

Because of the error in the data, no attempt was 
made to distinguish subjects who were using a two 
dimensional perspective from those who were using a three or 
four dimensional one. The question of "true" dimensionality 
of the solutions has to some extent, and of necessity, been 
avoided. Until more is known about the behaviour of Kruskal 
and Carroll scaling techniques under conditions of varying 
error and dimensionality, the problem of dimensionality of 
subjects’ cognitive structures will remain an open question. 

Gne suggestion which was tried and proved fruitless 
involves formation of subgroups based directly on 
personality variables. Subgroups were formed consisting of 


those one standard deviation above and one standard 
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deviation below the mean on each variable. Kruskal analysis 
of subgroup average data produced dimensions uninterpretable 
by the investigator. Carroll analysis of this data was not 
attempted. 

Returning to the question of the educational 
implications of the field of cognitive structure, it is not 
clear, despite the many untried and suggested possibilities, 
that the construct has any potential importance beyond 
theoretical interest. It is difficult to wax enthusiastic 
about the practical consequences for educational practice. 
This comment should not be construed as a suggestion that 
the field be abandoned, but only that the conceptual issues 
and instructional consequences be addressed at greater depth 


than has been done in the present study. 


5.4 Recommendations for Further Research 

This study has been exploratory in nature and its 
termination occurred at a somewhat arbitrary point. Although 
the questions posed have been addressed, the many fruitful 
avenues for data manipulation have not been explored. The 
study has shown that the recovery of individual differences 
in cognitive structure among subjects as young as the ninth 
grade is possible with relatively simple (for the subject) 
tests. To begin, techniques are needed for estimating the 
amount of error in the data. The Monte Carlo studies of 
Wagenaar and Padmos (1971) and Isaac and Poor (1974) have 


begun this work. The following studies are suggested: 
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1. Extension of the Monte Carlo studies to investigate 
situations where the true dimensionality is not exactly two 
or three dimensions, but perhaps an average of a true two 
dimensional and a true three dimensional pattern. 
2. Assessment of the various methods of introducing random 
error into Monte Carlo studies in order to investigate which 
is most similar to the type of error produced by human 
subjects. 
3. Monte Carlo studies comparing the accuracy of the Kruskal 
technique operating on an averaged solution with the Carroll 
technique operating over a set of individual matrices. 
Further analysis may reveal whether or not those 
subjects who failed to load highly on the dimensions of the 
Carroll analysis were operating from unique perspectives or 
from confusion and error. It is suggested that: 
4. Subjects who failed to analyse into subgroups on the 
Carroll analysis be regrouped and analysed again. 
The method used for subgroup formation may not be 
the most appropriate. It is suggested that: 
5. Alternative methods of subgroup formation, perhaps partly 
subjective, be explored in order to produce subgroups with 
identifiable aifferences in both cognitive structure and 


personality variables. 


6. Techniques of multiple regression and discriminant 
analysis be attempted in order to identify linear 
combinations of personality variables which will 


characterize subgroup members. 
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Further use of the semantic differential technique 
in the manner of this investigation requires exploration of 
the relationship between this use of the techniaue and more 
traditional uses. The probability exists that the scales 
provided for subjects to rate the concepts did not exhaust 
the dimensions of semantic space used by the subjects in 
their views of the concepts. Further use of the semantic 
differential in studies of this nature is of theoretical 
interest only, as the construct validity of cognitive 
structure has been established by the convergence of the 
constrained association and similarity judgment tests. For 
exploration of the semantic differential, it is suggested 
that: 

7. The data from the present study be factor analysed by 
methods common to semantic diferential research methodology 
to assess the adequacy of the instrument from the point of 
view of the semantic differential tradition. 

8. That the number of bipolar scales be increased in an 
attempt to produce better "coverage”™ of the relevant 
dimensions of semantic space. 

While it seems guite reasonable to extend the use of 
the constrained weaneeonsen test to twenty or more concepts, 
a complete similarity judgment test for such a large number 
of pairs of concepts would be a prohibitive task for a high 
school age volunteer subject. Research is needed on methods 
of deletion of pairs of concepts to shorten the test with 


minimal loss of information. It remains to be established 
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whether or not a method of discarding data based upon the 
relative sizes of the dissimilarities is more efficient in 
terms of information retention than a random discarding 
scheme. Such a non-random method would be based upon the 
fact that more information is needed about the relative 
Sizes of similarities of points closer together than of 
points farther apart. It is suggested that: 

9. Studies such as those undertaken by Spence and Domoney 
(1974), which investigate the effect of incomplete 
Similarity matrices on Kruskal solutions be expanded. There 
is a need for the investigation of the relative effects of 
random vs. systematic discarding of data. 

10. Studies such as those outlined by Torgerson (1958) 
involving the use of subsamples of the similarity matrix to 
estimate consistency of response be carried out. 

It seems quite clear that there are perspectives 
from which to view the concepts under study which cannot 
easily be explained by an outside adult observer. Given that 
it can be established that these perspectives are not merely 
random error, methods are needed for revealing the nature of 
the individual views of the concepts. Mass sampling large 
scale studies will probably not be effective for such a 
purpose. It is more likely that a combination of written 
essay (perhaps paragraph completion) tests and personal 
interviews will shed light on this area. Perhaps the 
training of subjects in introspective techniques would be 


useful here. The following exploratory studies are 
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recommended: 

11. Content analysis (Shavelson and Geeslin, 1975) of 
subject produced paragraphs explaining the concepts of 
interest. 

12. Triad studies such as those used in development of the 
instruments of this study. Subjects would be asked to 
indicate as many ways as possible that one concept ina 
triad differed from the other two. 

13. Exploration of the possibilities of training subjects in 
technigues designed to reduce error in their similarity 
judgments. 

The adequacy of geometric models of cognitive 
structure is not established. Some work has been done on 
non-geometric models such as hierarchical clustering (S.C. 
Johnson, 1967), tree construction (Shavelson, 1974), and 
lexical graphing (Rapoport et al, 1966). While these 
techniques seem to require extensive subject training, and 
perhaps must be limited to older subjects, their use may be 
necessary in cases where there is a tendency on the part of 
multidimensional scaling techniques to produce a degenerate 
solution. It is suggested that: 

14. Techniques Halekeieeda for making the method of tree 
construction clear to high school age subjects. 

15. Following the above, correlation bet ween tree 
construction and similarity rating techniques used in this 
study should be established. The investigation may be 


extended to situations where geometrical models may be 
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expected to degenerate. 

Wexler and Romney (1972) have devised a methodology 
for identifying individual differences. On the basis of 
subject responses, they calculated similarities between 
subjects. Kruskal scaling of the subject similarities then 
produced a map of subjects in geometrical space. Because of 
the general lack of success of the investigation in 
appropriate identification of individual differences among 
subjects, the approach of Wexler and Romney may prove 
valuable in more specific identification of subject 
characteristics with individual differences in cognitive 
structures. Perhaps goodness-of-fit measures on scaling 
solutions of individuals differentiated on several different 
criteria, such as responses, biographical data, etc, may 
provide insight into the type of variables best associated 
with individual differences in cognitive structure. However, 
such techniques, exotic and interesting as they are, may 
prove no more effective than conventional methods. It is 
recommended that: 

16. A subjects by subjects correlation matrix be formed from 
the vectors of similarity ratings produced by the subjects. 
This correlation matrix should then be scaled by the Kruskal 
technigue, and loadings on dimensions correlated with 
personality variables. Perhaps a discriminant function could 
be applied to the data to identify linear combinations of 
personality variables useful in differentiating among those 


with differing perceptions of the concepts. This suggestion 
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amounts to a nonmetric version of the Tucker method. 

The relative robustness of the Carroll technique 
with the type of data used in the present study may make it 
the preferred analytic technique for the further 
investigation of cognitive structure. This use of the 
Carroll technique has serious consequences concerning the 
cost of such studies, although no suggestions for further 
research can be put forward at this time. If multiple 
regression techniques are to be used for the identification 
of individual differences, then sample sizes considerably 
larger than those in the present study will be necessary. 
The computer costs of the Carroll analysis vary directly 
with sample size (and number of iterations). A run of the 
original INDSCAL program (Chang and Carroll, 1979) on 78 
subjects required, on the University of Alberta's Amdahl 
460/67 computer, about seven minutes of CPU time. Using the 
improved version, which became available near the end of the 
investigation, SINDSCAL (Pruzansky, 1975), this cost was cut 
im haift. 

Large scale data manipulation studies may not be the 
most efficient method of answering the questions which can 
be raised or cognitive structures. More varied 
experimental techniques are needed. 

In conclusion, there is the possibility that the 
mathematical techniques of the scaling analyses have become 
overly sophisticated in relation to the type of input which 


can presently be provided. It is expected that future 
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progress in the elucidation of cognitive structures will 
come only after considerable work has heen done on reducing 
error in the data and on clarifying and measuring relevant 


personality variables. 


183 


Aikenhead, Glen S. Learning about the scientific enterprise 
in_Saskatchewan high school chemistry classes: an 
evaluation. Information and Research Report, 
Department of Curriculum Studies, College of 
Education, University of Saskatchewan, Saskatoon, 
1832. 


Ausubel, David P. Cognitive structure and the facilitation 
of meaningful verbal learning. in Readings in the 


ee ese ee ee ths oe ee 


Psychology of Cognition. ed. by R.C. Anderson and 


D.P.Ausubel. New York: Holt, Rinehart, and 
Winston. 1965 


Bart, William M. The factor structure of formal operations. 
British Journal of Educational Psychology, 1971, 


ee re EE RE ES Se ES RS Se ee SE ae ee ee a ee 


Bieri, James et al. Clinical and Social Judgment: the 
Discrimination of Behavioural Information. New 
York: John Wiley and Sons, 1966. Pp 182-206. 


Black, Max. Models and Archetypes. in Philosophy of 


Educational Research ed. by H.S.Broudy, R.H.#nnis, 
and L.I.krimerman. New York: Wiley. 1973. 


Boersma, Frederick J. Test-retest reliability of the CF-1 
hidden figures test. Educational and Psychological 
Measurement, 1968, 28, 555-559. 


Braithwaite, Richard B. The structure of a scientific systen 
in Philosophy of Educational Research ed. by 
H.S.Broudy, R.H.Ennis, and L.I.Krimerman. New 
York: Wiley, 1973. 


Bruner, Jerome S. The Process of Fducation. New York, Random 
House, 1960. 


Campbell, Donald T. and Fiske, Donald W. Convergent and 


discriminant validation by the multitrait- 
multimethod matrix. Psychological Bulletin, 1959, 
56, 91-705. 


Carroll, John B. Words, meanings, and concepts. Harvard 
Educational Review , 1964, 34, 178-202. 

Carroll, J.Douglas. Individual differences and 
multidimensional scaling. In Multidimensional 
Scaling; Theory and Applications in the Behavioral 
Sciences Volume 1: Theory. ed. by R.N.Shepard, 
A.K.~Romney, and S.B.Nerlove. New York: Seminar 


Press, 1972. 


cut 


7 | . ees VOL IRIE 

sizajesne pidituedne sit “pagezeel .«t ool wheodaottA 
oe. a vcopatla viinineds 1eoda th cid. ae vedys 
, PTO gon dpipeAael. Paes ne. JemzoInl «id 


te senting  <zethnde -enkegramo 8 '° jiens 370400 
yoo tH Bee _shvadntataese, Te yiievovial ,nokreouhs 
SOT 


noitptetion? a4 bab oautoutie evitiapoo a Skved ,lecupas 
+ ct auttised of .poinzees Ledtev [otpainses 10 
sartmPn’ «20% 4G 268 emoitinpod io veolodausd 
ite _orr bons 4? Lak 4 470¥ we eladungk 4 be 
eaer .sovan te 


noeitszeqo ise10r 30 sivsorite r0OtoK? edT oF meiiftw ,2168 
Per sy potodoyes  t caoiseoubs to. Lesipot—_ ta 


£2. piane pbat {einer pes. £Reiatsd 2ifi2e eeget ,tzeia 
at gpodssuzotar _Lezpotvedos 30.8 


ant -chr o@ def yanoe ne yeili® adot :A70Y 
to ¥dvoget sd’ ni »eedytedotéA bas eaiebot xen 0806 Ae TA 
aad?" .? ,vhvotise =f ae .ho dorseset Ispoissauhs 

ap 


wwolr’ saAgOY wor SEM TOME. T.J Bas 


of io ytdlidetiaes teator-te9T .L #otwbett ,sme7908 
[eoipolod ava bus s_lenoitaoubs tues ae tppii nebbid 
sPEe~cee 38S ,80er ,JnomomuR sem 


solttueioe # 20 exmmdngase edt ati hisdo is ,ottewdt i624 
to dprsene® fsnoiseopbg Jo vtgosotida at 
tewrornTe. Ted ble ainda ef .Yiuos8 22 Hl 

ever ,yoli® sAtoY 


net \~2a0Y wor -oidpoubs 3o_ee99035_od% .2 owomel ,zenv74 
-Od0t avo 


fee neprevao? .¥ bisaed ,etett has .T bienod ,ifedqas 
=sfe77E tive ai? vi « potssebiiev tnanimitoeth 
0POf ,obsetiue L[evipolodoyed .xiaten bodsonijs ion 


bievssH .<wtaqaonos bas vopninsem 9Bhr0R .f atot forme 
-SOS-PTF Bf ,~oORT waived 


han ponmaratirt eerie ifert -eefpned .b , {for3689 
{png iguamihhadon nt ; 


tenolescnantanie 
~Prpaed? wet yd 
tpvinns 2: avoY we 


184 


Carroll, J.D. and Chang, J.-J. An analysis of individual 
differences in multidimensional scaling via an N- 
way generalization of "Rckart-Young" 
decomposition. Psychometrika, 1970, 35, 283-319. 


Carroll, J.D. and Wish, M. Multidimensional perceptual 
models and measurement methods. in Handbook of 
Perception Volume IT: Psychophysical Judgment and 
Measurement. ed. by £.C. Carterette and M.P. 
Friedman. New York: Academic Press, 1974. 


Clark, Andrew Kk. Re-evaluation of Monte Carlo studies in 
nonmetric multidimensional scaling. Psychometrika, 
1976, 41, 401-403. 


Cohen, Harvey S. and Jones, Lawrence EF. The effects of 
random error and subsampling of dimensions on 


recovery of configurations by non-metric 
multidimensional scaling. Psychometrika, 1974, 39, 
69-90. 

Cronbach, Lee De Test Validation. in Educational 
Measurement. second edition, ed. by Re toe 
Thorndike. Washington? American Council on 


Education, 1971. 


Davis, J. Kent. Strategy development and hypothesis testing 
as a function of an individual's cognitive style. 
Final report, U. S. Office of Education Project 1- 
E-O067 (ERIC #071010), November, 1972. 


DiVesta, Francis, and Dick, Walter. The test-retest 
reliability of children's ratings on the semantic 
differential. Educational and Psychological 


Evanechko, Peter 0. and Maguire, Thomas 0. The dimensions of 
children's meaning space. American Fducational 


ee ee ee a ce re eee ee ee ee ee ee ee ee 


Research Journal, 1972, 9, 507,523. 


Ferguson, George A. Statistical Analysis in Psychology and 
Education. Third edition. New York: McGraw-Hill, 
1971. 


Fiedler, F. A_Theory of Leadership Effectiveness. 
McGraw-Hill, 1967. 


Field, T.W. and cCropley, A.J. Cognitive style and science 
achievement. Journal of Research in Science 
Teaching, 1969, 6, 2-10. 


Fleishman, J.J. and Dusek, R. Reliability and learning 
factors associated with cognitive tests. 


var 


ruuhivenet yo \Seeeheae “ae SeGs0% rede Bae 0.6 \Ilorzed 
ay ne nav. ponhese * Enaodggiom tha hve a2 edoneie82 ih 
Hem ay=t tHtor" Pan pe epee nO vee 


OTe=Es 4c€ , OTC? sabbatamedavet aot 


feu taenye4q tego ae aes 2 6atetl | ohne <2.  Ho2zse9 
1 sogdsust ni hii eae hos alebou 
ie 18 vi 


fi djnenphul 
1 — ser § cha ao1Bsh08 
<aunr? cigeseur swroY wet .nenhet 


SI NF ’ 
‘ 


nf o@tbyu+e off) O2noOF To nosteuleve-of <% westbaa etm 19 
bs tn oot sya? -patisoe2 fenofeznemtbhitiom sivy'ennon 
oF u-roe te yater 


tn «6c 2 tO ed? 8 opnetvaAl ,2onot foe «e yovTsl ,nodod 
sa cteyetonth to patiqapedua fae 20719 sobast 
jiv) ob snOn yd pnol + BYePL THOS toe yievonet 
«' “urer ,vdiytewodoynd spatisoe faaotnoont bt tiva 
-OP-ea 

ienvisenu® ai rotrohi ley feorT of 991 yiosdnot) 

ae i “De wo ftibe hrones signe ive5eR 

a Fine: opty trem A sdorpaidas® -Orthbawodt 
.tver. ,nolteoubs 

nirctee? odtoavd bar *+nengoLlavelb Ypstarte snes «t- ,2ived 
soy ¥: wit topos ntfauhivibat os Yo foticoant & @6 


_f} s4ere7? nobtenebY Yo eof720 .e «U0 .tI0qGe3 [ent 
eSTer .redme von -(ororros 2795) TaO=d 


tpatot-p>o 4 ont gotiew ~ goto bos yeionsad ~staeViG 
sitenane oft a0 apattet a *snearbiino to yitilidsiies 
tesignlotoyst 2 45 hos. _fenes? -leitmeveriih 
aT o-e00 ,9S ,sdeT ,InomerBesed 

intenemib od? .& neaod? ,o Tie en has «9 Weted , ot doe wavs 
Lanetteogh! aegtieds .eoBge painsom a'tnethiido 
foe,coe ge ,SVOT ,Lenauel. dogegeos 

bae ..vuolodoyes ui slaytE £2 +2 .f Opto9d ,oenpm94 

cit ia-wh oon sia0Y wel .wO89 a1idt 
af 

:870y wet ,~pReneys .j29 mm. Pe i .wihstt 

ed  ffLit-ws 2008 


vainasel ban ydikidstiet A ,aeeud hae 
wad avitiapos dtiw hetetooran 


185 


Psychological Reports, 1971, 29, 523-530. 


French, J.R., Ekstorm, R.N., and Price, L. Kit of Reference 
Tests for Cognitive Factors. Princeton, N.J.: 
Educational Testing Service, 1965. 


Gardner, Riley W. Reliability of group-test scores for 
cognitive controls and intellectual abilities over 
a one-year period. Perceptual and Motor Skills, 
1973, (36, -753-754. 


Gardner, R.W. and Lohrenz, W. Some old and new group tests 
of cognitive controls and intellectual abilities. 
Perceptual ad Botor Skilis, 1969, 29, 935-950. 


Gardner, a Be and Schoen, R.A. Differentiation and 
abstraction in concept formation. Psychological 
Monographs, 1962, 76, whole # 560. 


Garskof, B.F., and Houston, J.P. Measurement of verbal 
relatedness: an idiographic approach. 
Psychological Review, 1963, 70, 277-288. 


Hammond, Allen. Assessment context in relation to physics 
achievement and cognitive style. Unpublished Ph. 
D. dissertation, University of Alberta, 1976. 


Harary, F., Worman, 8.2., and Cartwright, DBD. Structural 


Nodels: an introduction to the theory of directed 
graphs. New York: John Wiley and Sons, 1965. 


Hartnett, Barbara M., and Rumery, Robert E. Markov Chain 
analysis of classroom interaction data. A paper 
presented to the annual meeting of the American 
Educational Research Association. New Orleans, 
1973. 


Harvey, C.J. Conceptual systems and attitude change. In 
Attitude, Ego-Involvement, and Change. ed. by C.¥. 


Sherif and M. Sherif. New York: John Wiley and 
Sons, 1967. 


Harvey, O.J., Hunt, D.E., and Schroder, 4H. Conceptual 
Systems _and Personality Organization. New York: 


John Wiley and Sons, 1961. 


Heise, David R. Semantic differential profiles for 1000 most 
frequent English words. Psychological Monographs, 
1965, 79, whole #601. 


Heise, David RP. Some methodological issues in semantic 
differential research. Psychological Bulletin, 
1969, 72, 406-422. 


af e-PSe BSS oe e2s0005_Lortpotodaye 
auars}o2 36 ei wt cob te Pass yell oh aot «Mat hones? 


oe a a 


-.,.¢ fotos) sapegoet oust iano? —77 a7. aster 
ear sookwie piLt290T [snoi t600 


se% aeqooe faot-atiotp Fo ytilidsiios «fF yeti? , 790b160 
save maitifide €nvtoelieta bin miovtnos evitinpes 
spblitz..zorot. fas. teaieeores « «shotzaq IseYy-so’ s 

.wet-fev ,of ,E Ter 


+207 spot wer hae hilo emo® oF ~saetdod fas oH ef . tenh269 

wattifidn (enol fo tab tow elorza02 evttiapos 10 
aav-ary .2¢ ,Paet sahbbde totem bas Leutasors) 

hug ito beef tome ettic iA? ,nsad 2 hos «W598 , tenbhaH9 
lpoipofodoy s nor eee, tqenn0D at not topztads 
ja) © efodw gat .S8e ,edaszpono’ 

‘agev to newno TH Cc War feb hada ba 6 ee 9.8 ,o#2769 
stoeegrings nidaetmporbt sanonhetsior 
sBPLHTNS 4OR a Seee cusivoll_lenmpolodoen’ 

ue wits ot soseetor ot sos nop. sanmaeoesh -noltA = ,fhaormsh 
. tadrilduqal sole de Svesiapoo. bas jneaevelson 

ater oerortta di vitetovian ~noiteszeael 
ter soepse «tl qg tip haw? 26° hae yeSad) «,nemtOw 4. «yy IeH 


hetoeyih 70 Berane “pat od. noitovhogtai op selohon 
ener v2aor Sin vyale® mfol sitoY wot sodgbap 


} vty ‘i 


9 ¢7rates .yteauh bos yh ersadye®  ,ttent 76h 
reget &  nteh nolbtoasetasr nooreazs!(> to atavisos 


ientzvag’ edt 40 ontteow Beunnh edt 07 Letneae2q 
eieet?o wat ,notteicoseé foapees? LIsnortsouhs 
-£00f 


panda shotitts hos  saereye {[etqoonod 20.0 ,yovish 


4.) vi .be .opoedo bop. stagmeutount-opd 
yvati nado st3eY von .tttod2 .m bas YL196 
.Taer enor 
lausgeodod «Hf ,tehordo® Ba6. 468 <0 ino «6466.0 ©«6yevIBe 
oy watt siobtexioneie seebpaegie.-2 | 
: «fa +. no2 bas Y : 


-+ y QUO 4o% Relttots Pedtdete®®kh oitonase <2 biveso smi? | 
weicernavot Leotpolodoygg ae daitpat tne 
.foat ofodw .ot 


~“tiaopeee ni Boaveel {62 inolobodiom enor a biveo 
wisgliud. Lestpofodoyad aides oe Td 
rs 


we 


186 


Hobbs, Edward D. Formal operations in secondary students: a 


test___of the idea of intellectual structural 


limitations. Unpublished Ph. oD. dissertation. 
University of Alberta, 1975. 


Holzman, P.S. Scanning: a principle of reality contact. 
Perceptual and Motor Skilis, 1966, 23, 835-844. 

Hunt, David FE. Matching Models in _Fducation: __the 
Coordination of Teaching Methods with Student 
Characteristics. Monograph No. LU Ontario 


Institute for Studies in Education. Toronto, 1971. 


Inhelder, Barbel, and Piaget, Jean The Growth of Logical 
Thinking from Childhood to Adolescence. New York: 


Basic Books, 1958. 


Isaac, P.D., and Poor, »D.S. On the determination of 
appropriate dimensionality in data with error. 
Psychometrika, 1974, 39, 91-109. 


Jackson, 0D.N. and Messick, S. Individual Differences in 
social perception. British Journal of Social __ond 
Clinical Psychology, 1963, 2, 1-10. 


Jackson, D.N., Messick, S., and Myers, C.T. Evaluation of 
group and individual forms of emmbedded figures 
measures of field independence. Educational and 
Psychological Measurement, 1964, 24, 177-192. 


Jakobovits, L. Comparitive psycholinguistics in the study of 
culture. International Journal of Psychology, 
1966, 1, 15-37. 


Johnson, Paul E. Associative meaning of concepts in physics. 
Journal of Educational Psychology, 1964, 55, 84- 
a0. 


Johnson, Paul &. Word relatedness and problem solving in 
high school physics. Journal of Educational 


me ee ee a a ee ee 


Psychology, 1965, 56, 217-224. 


Johnson, Paul &. Some psychological aspects of subject 
matter structure. Journal of __ Educational 
Psychology, 1967, 58, 75-83. 


Johnson, P.£., Cox, D.L., and Curran, T.E. Psychological 
reality of physical concepts. Psychonomic Science, 
1970, 19, 245-246. 


Johnson, Paul 8, Curran, Thomas E., and Cox, David L. A 
model for knowledge of concepts in science. 
Journal of Research in Science Teaching, 1971, 8, 


AGS 


» se sahy sy 
coe hae 
.804%R Tyee 


:: 


*aetnog ytrtiee 34 siqiontia9 NK enn tamer | , noms lok 


cinneete ff 8000 ,phidae autos bas Leusegoues 
oi tenuhs wall Sit eC T * bhived tau 
y tienibzeo2 


Aa eet grade 
otis it 1 et ery 222 ‘Lop Eaesoeaado 
oft et .,aaner0? nolL eo0b3 wt aeifuse 102 otnstte 


ol fo dj#ozh ght neotl ,*epert bas ied 154 iehiedal 
joW 2opngogsiok 3 42 mo13_posans se - 
Reet ,etoon oiess 


aotisntaqoter att a0 -ses0 ~roo" «bab yA q ~oaeal 
swe tfiw ate as vtifenoianomib ofsitqorgqs 
.eor-re sek ,ever sstinz 


t Di poAN ‘eubkvihat .@  .#oteeeM bas 4.0 0=— , oats 
bao Leloo? 1o_fuaqbeh tek aeae .nottaeoieq [stor 
OF -T ae sypofodoved Isniai 


yi o%.D ,Rzauh HME, « of ~ioiaaot = y-4. ~noatosl 
apPhodege Yo asyot Isphivibat fos quomp - 

sh? epgofngoehark. blett to aetue £oR 

- 


ox ,dnet .isemoau2por_isotvolodaved 


vbuta ode of soltednpatiodsyag svitizrsane? wf ,adivodoist 
,eelosove? 28 __{eogivol Lemolssazesat or tino 
avi-et yf ,aaet 


s¢oyde ak steangen Bo pitisen avit+sipvoned «3 Lved , aozndot 


we ie yvaet ,vpoodoyeds Leet tepubs__i0__Leaauek 


. oneviar metdetg fre meenheseley bro¥ .s Tues ,moandot 
-oiteouht to. Lacayeh westeydd | looms dped 
oes =TES oe scart s¥ 


iseraie io efceden Inotpalodoyer onoe 2 Insd ,soenadot 


Legu £S0QUn8 _ ~28... -Leuiesh - rnc erdon 
wet 4Be « 


feed tel odovad | stu aia har «1 .xOD 
wungaio? obwonodpye? .aranonee tsaiavy 10 yikes 


A .? fived ,xeD Bae sou aseodT |, 
onan nt pois pptices to. h 
» «OT shaddose ratoe ot 


pe7I0D 469 
7 w 7 ¥ 


187 


o1=955 


Johnson, Stephen C. Hierarchical clustering schemes. 
Psychometrika, 1967, 32, 241-2548. 


Kaiser, Henry F. Relating factors betwen studies based upon 
different individuals. Unpublished document. 
Bureau of Educational Research, University of 
Illinois, 1960. 


Kaplan, Abraham. The Conduct of Enquiry 
Chandler Publ., 1964. 


Scranton, Penns 


Karplus, R. and tLavetelli, C. The developmental theory of 


ee ee cee ee A aD em ee re eee oe 


a eee oe ee rm ee ee ee 


physics concepts. Unpublished Ph.D. dissertation, 
University of Alberta, 1969. 


Kass, Heidi. Structure in perceived relations among physics 
concepts. Journal of Research in Science Teaching, 
1745 yg 339-300. 


Kelly, George a. Psychology of Personal Constructs. New 
York: Norton Books, 1955. 


Klahr, David. A Monte Carlo investigation of the statistical 
Significance of Kruskal's non-metric scaling 


eos 


Kiein, G.S., Barr, H.-L, and Wolitzky, D.L. Personality. 
Annual Review of Psychology, 1967, 18, 467-560. 


Kruskal, Joseph B. Multidimensional scaling by optimizing 
goodness of fit to a nonmetric hypothesis. 
Psychometrika, 1964a, 29, 1-27. 


Kruskal, Joseph B. Nonmetric multidimensional scaling: a 
numerical method. Psychometrika, 1964b, 29, 115- 
123. 


Kruskal, Joseph B., and Carmone, Frank. How to use M-D-Scal 


cere ee Se ce ee ee ee 


Telephone Laboratories, Murray Hill, N.J., 1971. 


Kuhn, Thomas S. The Structure of Scientific Revolutions. 
Second Edition. Chicago: University of Chicago 
Press, 1962. (1970) 


Lawson, Anton. Implications of Piaget's theory for 
presentation of content and process. A paper 


rer : - 


@ 
1 : a ay 
: <ca- r 
- 


ty = ad. 
aeretie «on Peete © Samper eae a | sd wodgete ,aonadol 
eet =TNE ‘£6 , Tae yhz 


Ligkod a4 : Shia ise ist 


aggt foeed BoLhe ze | usy. 
. moaenranot ' — thrtveet. ; 
be et imag aris Rr ok! Bd tendo aoube yo ueevon 


.ooet ,atowiiit 


sot  notenio® <yphupee ie _spubso2 ssf ensdeidaé ,anlqen 
: wart ,.fdn? relhasdo 


vw vvoed >> bsges ago fevebiedt 69 yi iotove? hia of  ,aulqrer 
hign®? ato ifica Toanhy? W652 ) 3 be] 
Ses m 

pitta sand tate 4 owniane a6 oso ~-2aws gus te pAT .rbLo# y2n5" 
. 20 17h P72 “aT? Aa teint d snigsnnon _aatayda 
.oaet qetied tA Yo ytiarevial 

~poven neem ano ftatet hayfen eq at oiptoutte efhiel ,2eBn 
putdgeet Sh a ne eet qanago 
7 | “Der -OFfE 48 ,tter 


lo 2 ¥ | - epio90 «6. ELon 
«eee? aces not ror s410¥ 


temite teat off 26 sob +apheeennt atte? staoF A «hived vais th 
ctu spy tom =701 e'(spaeute to ennsok i tapie 
ly rlere gE Oe? gudigzomodoyed .67uhes0% 


ifnnentad ale gas *hto¥ hone ot. el ag -e2em = tolt 
Loe'-Ta9 01 .Sa0t supe lodpya’ 2o_well 


ovntederea vr poitss beaphanenthasiut 2 dgonot ,iseteui4 


ageeiioayt oitteanon £ 9? +t? to eae no 


«6 «porbleor Letotenempbraiia oFrdsanor 4 , [stauw 
<r nee <2e00T efit tameidgyad ~~ sbodjes fenivomoun 
St 
lent -0-B. onto? sol) + ,isdteut 
Clot sgodtsazoigt_ivis wy rosso bos. 4 
ATE ieuae gy Cnet vp rTP eee ties , R 


sano dap hoves colts not oe 
epenid? ta. ytharvevin’ 


19 yrondt  attepeit 20 nop stoi laut, sande orwed 
tocen of s@eenoz7a ban tnotnOD" i 


188 


presented at the annual meeting of the National 
Science Teachers Association. Philadelphia, 1976. 


Lawson, Anton E. and Nordland, Floyd H. The factor structure 
of some Piagetian tasks. Journal of Research in 
Sclence Teaching, 1976, 13, 461-466. 


Lawson, Anton E., and Renner, John W. A quantitative 
analysis of responses to Piagetian tasks and its 
implications for curriculum. Science Fducation, 
1974 ~ 58, 545-559. 


Lawson, Anton E. and Renner, John W. Relationships of 
science subject matter and developmental levels of 
learners. Journal of Research in Science teaching, 
19795, 12, 347-358. 


Lingoes, James C. and Schonemann, Peter H. Alternative 
measures of fit for the Schonemann-Carroll matrix 
fitting algorithm. Psychometrika, 1974, 39, 423- 
G27< 


Lohrenz, Leander J. and Gardner, Riley W. Cognitive control 
in recall of similar visual designs vs. nonsimilar 
thematic materials. Perceptual and Motor Skills, 
1973, 36, 621-631. 


Lunzer, Fric A. Problems of formal reasoning in test 
Situations. Cognitive Development in Children: 
Five Monographs of the Society for Research in 
Child Development. Chicago: University of Chicago 
Press, 1970. (original publication, 1962) 


MacDonald, John. A_ study of the relationship between student 
achievement of process skills and the mode of 
instruction in junior high school science. 
Unpublished Ph.D. dissertation, University of 
Alberta, 1974. 


Maguire, Thomas 0. Semantic differential methodology for the 
structuring of attitudes. American Educational 
Research Journal, 1973, 10, 295-306. 


Messick, Samuel J. Metric properties of the semantic 
differential. Educational___and Psychological 
Measurement, 1957, 17, 200-206. 


Messick, Samuel. The criterion problem in the evaluation of 
instruction: assessing possible, not just 
probable, intended outcomes. In The Evaluation of 


Instruction: Issues__and Problems. ed. by M.Cc. 


eee 


Wittrock and D.E. Wiley. New York: Holt, Rinehart, 
and Winston, 1970. 


Ont 


fanoitcokt oc? To pabiome favean ect a hesneecrt 
aver .ntdefehalidt - lee fooa ed avaiune? aenne@ioz 


e1vsootte tofon! ef? <i hveoLt ehtia lh 20% tap «8 ROSnk ,notwel 


of doreeuet. to Lasqeoh erase? avitersr? 98or 

a (baw -rae GEL yeTeh .poldope? epmehae 
withisenouer «#8 = sot <Temed == he ~~ noeoOA ionwbd 
4} ton o2204 sehr topald -O? aaadeges: to )6€GC efayiens 

notsnvuh? sagedas eulusiaio> 103 enokteotiqat 


028-202 .8e ,Oter 


re: cop tus moet alee 8 «dot. «rennet fas <9 notak ,norwal 
in afavel Loetaenanfeveb fas Jazze toordue ennbion 
wilidpes* opnoint at dopeegen 29. yiascid -vton tool 
THRE ght , eve? 

vitenSaetA 2 S788 ee bas .) eeant ,aeopaid 
‘ew Lflorten-nnamaenodn? eh? to? 47+? to GBetUnSeOD 


5 ck «6 ever ,etta somodsyes edt inopls — 


Peovw tewes) yittera) | Y° ie ay yrPahtsAH bos .f. yeh teal ,snetdod 


eTigtoace wow enpteeoh Letnie DPaliaste. To {Isoe7 at 
etlize..s0t08_ bos. Legsaso ged <eistroipe Sitnnedd 
-fEa-TSa ,GE Ever 

oiizoneog Lento? pmafdor? .«A of3% ,tes002 
-t>ltuy. ai  2pegqn taveg. 2 exksiagen " -aro}se0ete 

=) J0oe0500 303 wae igee oat 

as ra” ‘iavavind :ophoid] jaaesolovenEikta 
(caet ,sottsotideg Isnipito) fer ,szee%9 


toohyte gooey jeg obdagolteler odd 20 vbute £ ngot . 51 snodo68 
{c_ show ol? foe. sfbise epagoaa. Jo" ioemocakdas 
~ojusioe _. logdaa 4 pis. yosgut ni. notionsegs 
to) 6 VA rev pa Vlott etiemetb «0. AF hedai idnqal 

over ,stiuedtA 


+ sat yootobod+em Letiqaaetsfh 2i¢neme2? .0 oamodT ,otiupeh 
Lashsnsnee. HL L189 coafutiiia to wala ioeee 


al 


Lauf-e?% OF ,ONOP yLearwo 


‘itaneae ett 2et di 0 jor. of Ion »tolaazokt 
it pepo iadaye ad Rene 6 NOLTHD I 2 «isitnet ‘26 


to gobterleve ed? st wesfdorg. rk ae edt ab viotagot 
raeito 


ter ton <aidsazod par 
lo _aodseuleys e82 «1 

pe b's,» yd ho een oe! 

ae a, ae <Pion rta0Y con ae 


700809 80 


189 


Messick, Samuel and French, John W. Dimensions of cognitive 
closure. Multivariate Behavioural Research, 1975, 
10 5. 4-16. 


Miller, George A. The magical number seven, plus or minus 
two: some limits on our capacity for processing 
information. Psychological Review, 1956, 63, 81- 
97. 


Morgan, Kathryn P. Jerome S._ Bruner's theory of knowledge: a 
philosophical critique. Unpub. HEC. thesis, 
University of Alberta, 1972. 


Norman, Warren T. Stability characteristics of the semantic 
differential. American Journal of Psychology, 
1959, 72, 581-5248. 


Osgood, Charles F. and Suci, George J. Pactor analysis of 
meaning. Journal _of Experimental Psychology, 1955, 
20, 325-3338. 


Gsgood, C.8., Suci, G.J., and Tannenbaun, PH. The 
Measurement of Meaning. Urbana, I111.: University 
of Illinois Press, 1957. 


Preece, P.F.W. Mapping cognitive structure: a comparison of 
methods. Journal of Educational Psychology, 1976a, 
68, 1-8. 


Preece, P.F.H. Science concepts in semantic space-a 
multidimensional scaling study. Alberta Journal of 
Educational Research, 1976b, 22, 281-286. 


Preece, P.F.W. A note on the comparison of cognitive 
structure and subject matter structure. Journal of 
Research in Science Teaching, 1976c, 13, 351-352. 


Pruzansky, Sandra. How to use SINDSCAL: a computer program 
for individual differences. Unpublished document. 
Murray Hill, New Jersey: Bell Telephone 


Laboratories, 1975. 


Rapoport, Amnon et al. A_ study of lexical graphs. 
Foundations of Language, 1966, 2, 338-376. 


Raven, Ronald. The development of a test of Piaget's logical 
operations. Science Education, 1973, 57, 377-385. 


Raven, Ronald and Polanski, Harold. Relationships among 
Piaget's logical operations, science content 
comprehension, critical thinking and creativity. 
Science Education, 1974, 58, 531-544, 


ont 


avetinpos Yo #nefseorrn 2M nde) ~dovest han Louw he , to leaeh 


yer et wingesned Perot ties yezeves ies -oreao ID 

2 atat 4 Ot 
annie th avid .€ oreo) ToseER favirat “iT A SpTesd zo iL in 
gn ramanets o* 6 ytisecio 36g nA ~+intt som 30M? 
=f) fa. ,aret soive?. tsagpetodpyet «od fala se 
_sspbhefyoas. 20. vaends geyemuse. 2b esere of aytdtod .aspzos 
_~2aeks suns -dudnt saupitt 


ver ,sttedi’ to yttaszoviad 


—~-enepes O84 Ie i tePrethewedo ytritdes? .T ae 77Ts¥ 8 ,asmM70n 


-4.__ jo. Lsnqgoh_ mes iaous e[nisne tot? ib 
bne~f8e .St Peet 


isyferh JOPORT oC ap 1649 qfave Bas oF aelrsd2? foopead 


scurt ieolodoved Legdemegedua To isarvot -painsem 
sBEE-CKE 402 


B59 oad nearest Ait  yeG.t ,iov?  »e8.D hoop20 


foswin! +117 .sreda0 <paigeet 20 ssederueeel 
-faeF ynaert etonilft to 


ve osdeyase> # satpsontte ev iiinpos prigaeh .W6te9 yooesa TT 
avo? yvoofodoynd Epmokssoubt te Iecagol .sbodtem = 

-8-f , 80 
-,oeae oflosmer nt =?ganne> BUN Le oh -% 59 yeose TF 


ty Ieaynoh pied ta .yGote patiaoe Isnotenomibirlue 


“FOeATHE 4S WdaTOT sdpmpezed Senokssouby 


ittnpe» lo soabasqeom ad? mo sor A .#.%9 yonee74 
(anuet WI tonie yo¢tem ftockdee hos s1vtdourte 


syrtar ,(t .oatOh spaidoset sogetog of foresees 


eoipnyt teduenop 6 30 0080NT2 seu os woe -othae® ,ysenssu7d 
, *asanoonr hadadidvant . ash foptivibai 193 
njdeae tar fien eaanee walt elit Ne ey | 

eter ,asirotetodsd 


soe B Ts leorxal ‘o)60 so hote «6A Cet to 6=6(nonma yt Toqogs# 
-ITE-REE So .d0CP yopegpmps Jo. gnottebaued 


cnipel eltopert 26 +20% & Po taamqoleveb edt .bfisnof ,sovel 
PONE SOTE 52 CVPRS ,cotsepubs songize -2noitBh19do 


naoee onidanoiasie® blown’ siot bas Bipnof 
tsathos saedsine  ,sokteaego” tpol ettephid 


a 


sewed 


vrtetteeta bnew paitatit & 


.eue-ree she ete 


190 


Ross, John. A remark on Tucker and Messick's "points of 


Satterly, David J. Cognitive styles, spatial ability, and 
school achievement. Journal of Educational 
Psychology, 1976, 68, 36-42. 


Schonemann, Peter. A generalized solution of the orthogonal 
Procrustes problem. Psychometrika, 1966, 31, 1-10. 

Schonemann, Peter H. and Carroll, Robert M. Fitting one 
matrix to another under choice of a central 
dilation and a rigid motion. Psychometrika, 1970, 
255 245-255; 


Schwab, Joseph H. The concept of the structure of a 
discipline. in Conflicting Conceptions __of 
Curriculum. ed. by E.W.Eisner and E. Vallance 
Berkeley, Calif: McCutcheon Publ., 1974. 


Scott, William A. Cognitive complexity and cognitive 
flexibility. Sociometry, 1962, 25, 405-414. 


seaman, Jerrol M., and Koenig, Frederick. A comparison of 
measures of cognitive complexity. Sociometry, 
1978, Si, 375-3990. 


Shavelson, Richard J. Some aspects of the relationship 
between content structure and Cognitive structure 
in physics instruction. Unpublished Ph.D. 


dissertation, Stanford University, 1970. 


Shavelson, Richard J. Some aspects of the correspondence 
between content structure and cognitive structure 
in physics instruction. Journal of Educational 


cee ee ee eee 


Psychology, 1972, 63, 225-234. 


Shavelson, Richard J. Learning from physics instruction. 
Journal of Research in Science Teaching, 1973, 10, 
101-111. 


Shavelson, Richard J. Methods for examining representation 
of a subject matter structure in a student's 
memory. Journal of Research in Science Teaching, 
1974» 14, 431-258. 


Shavelson, R.J. and Geeslin, W.E. A method for examining 
subject matter structure in written material. 
Journal of Structural Learning, 1975, 4, 199-218. 


Shavelson, R.J. and Stanton, G.C. Construct validation: 
methodology and application to three measures of 
cognitive structure. Journal of Educational 


oer 


ad 


to atlniod™ attolzaet Pat ge ivw® no daaeoa, se arto’ anaon 
ofh-FS ghh pane gekeae sua suk gery 


Ann ethitdn tejtwrm .seiyae avitinpe? = .& bived a isortae 
snotteou he 4. 12st athetovertAcer Toots 
es cisar .Ba over ,vpokedaved 


teqonndtao sa? Ya aotsutop fasifegrsaaop A «Fete? ,ausuenodae 
«ht=-t <ft _baet seitzrogodoyet ame ldordq antag tos 


pricey .® #eedet. . hho rsso hie «A tete49 .o8asnodo? 


inytren .& 40 Stod> qebig Jedtons of xiatos 
corer ,etbasegodoyved Se hipit & has aoissttd 
. -PeS-2Bs yet 
. to werisodte ett 36 fosoo> of? .f daseot = ,inwdoe 
so. sogsarapo?__paggonl | ni ontiqioetb 
stelle’ .5 feb ToneLT.e, yd «hoe 


oun Luptz Hd 
ohTOr ¢sfdus noanoduooe ¢tpied ,yolortzes 


pianes rs viixelanon avisinpod — A mpilliv¥ ,ttonve 
oro-208 2S ,S80r gyahemodooe -ytilidixed 


-ryeoe # .aorvebeg* qpkaeot bas 4. eM for19G ,fanse? 
a Lizeke £202 .yttxelame> eyitinpos to 2xeieKeS 
.oPe-eve ,TE eter 


~ 


‘idnneitning pas - Jo 8 eipedgn ager «tL baadpie ,sonlevede 
Sou topsae avisiones. Las orp23uam samme eS aa 
is iaiciiows! <@ossoutsens, es | 

“OVOF Ve iscavint! Brolnste ,nolt+n tear th 


onsen off Jo atogqee aot .0 bredote ,woafoved2 
was) eri ap ny ing ovetoeate tnetnoo neewtod 
eault 2¢_ ~Lebages snoktonttent anteyda at 
HES-2S8 g£9 «Stet ,ypolodoyad 


vo t#ont tent. enobaydg nort potnaseld of, freddie sorlov ade 
OF sFUO) .goid ose? Some Bae rede 3 


ef 


‘ytaaeotaet bainipexe tot ahotter .& brednid ,nowleved2 


i 
~\irebhote fb 02 amisette deste toorduea 8 to 
jEaisneet.. sonese semen 


rniatenye To} battom A tah. gah 
loieerne oettiow of owt apt 
PPG POP gi ORR? - spa togas 


-yobtnhiley gnouwieaod <.8- 
*o Pett eee baie’ ot notte 
fesoit spake 


191 


Measurement, 1975, 12, 67-86. 


Shepard, Roger Ss The analysis of proximities: 
multidimensional scaling with an unknown distance 
function - I. Psychometrika, 1962a, 27, 125-139. 


Shepard, Roger ae The analysis of proximities: 
multidimensional scaling with an unknown distance 
function - II. Psychometrika, 1962b, 27, 219-245. 


Siegel, Sidney. Non-Parametric Statistics. New York: McGraw- 
Hill, 1956. 


Simpson, Ronald D., Shrum, John W., and Rentz, R. Robert. 
The science support scale: its appropriateness 
with high school students. Journal of Research in 
Science Teaching, 1972, 9, 123-126. 


Smith, S., and Leach, C. A hierarchical measure of cognitive 
complexity. British Journal of Psychology, 1972, 
63, 561-568. 


Spence, Tan and Domoney, Dennis. Single subject incomplete 
designs for non-metric multidimensional scaling. 
Psychometrika, 1974, 39, 469-490. 


Spence, Ian and Graef, Jed. The determination of the 
underlying dimensionaltiy of an empirically 
obtained matrix of proximities. Multivariate 


wee SS ee ee 


Behavioral Research, 1974, 9, 331-341. 


Subkoviak, Michael J. The use of multidimensional scaling in 
educational research. Review of Rducational 


eo ee ee me ee ee ee ee 


Research, 1975, 45, 387-423. 


Tanaka, Y.A. and Osgood, C.£. Cross-culture, cross-concept, 
and cross-subject generality of affective meaning 
systems. Journal of Personality and Social 


Psychology, 1965, 2, 143-153. 


Torgerson, Warren S. Theory and Methods of Scaling 
York: Wiley, 19538. 


Triandis, H.C. and Osgood, C.E. A comparative factorial 
analysis of semantic structure in monolingual 
Greek and American college students. Journal of 
Abnormal and Social Psychology, 1958, 57, 187-196. 


Tucker, L.~R. and Messick, S.~ An individual differences nodel 
for multidimensional scaling. Psychometrika, 1963, 
aay S43-307. 


Vernon, P.F. The distinctiveness of field independence. 


net 


* 


vaeite ah aevet <jaseorpanoe 


‘ é : 
‘nat eiwtyord *) i tae Cane a9" am 19pGR ,bracente 
wnjete kh awonten ae ooh Welipon fecotsaemtbhrttre 
Pt—-0ey, tS Td" sehen see ~T = noivone? 


: oj, ekyors ie tov fone oil'T ry T9p08 »brsqed2 
oaeeteth neordany un (9 to pataiage fanoinnonthriive 


Tavc-Ore VG .dcoe) -Aiksatefodoyad «fT ~ aottoail 


wnTON. AFOY wae w2pbdebtege of7% p28 10dzg0" -yonbie  fepeie 
weer Lf ee 


2 gk treat dedg 9 <¥ adot ~avade ..0 hisdaow ,nongate 
rotpgruovaan ath selsoe Poaqus anoieioe sad? 
Apaspeed to Isatmey etnebute foodoe dpid div 
OST=EST g@ gRUOT spuidoeet soneine 


igi* tapes oO ithe {snidpveterd A 2. ~ioseal Poe .e2@ tt rae 


\crer ,ypofodoved to_ Lemaiob deitizes «ytixolqnos 
fae-Pae 6a 


or. 
a) 1 ew ee 


tamcnan) toetdua ofpak? -einaed ,yonowod bos a67 ,eoneqe 
ifase (Ceooten as thtdatem srfatont-n0n 407 anpraoh 


Oona ,ef ,aTPT sbiirisnodovad 


ia Ws witeninteieb ‘ett. «bet e679 fos nest ~eonnge 
vifrorviqne, m6 to yitisnobenomth paiviaebas 
e*attevitiv? PR to xtittse benistdo 

ert ee sf OTOP adozeezed Leroivedos 


fi pedfenos fspobeneebhi+inw Yo e20 eft .6 lesdoie ,Asnivordve 


{eno typpuhe _ 30... wekves edot68on 81 fenot +soubo 
ESU-TRE ych yeotel sdozReRee 

, fqesdgn “2407 owt Lip-ne08d re) shoop20 bas .AaT sAeasT 
ptvses of Ad. 10 Lee pate t Harr a age bits 
Luizoe_. Sus. ~iilenosgad te —tee <amoseye 
STEEP ak Par *eepelodave’ 


é; stuifeot to shod sos. bas _eaeee® 2 netze¥ ,Mnoa70pIOT 
: r .yeLiw saa0Y 


rpisotor? ovEdsteqao> WW eed. gbooped (bab .OaR ,ribasitt 
(nepafieoss al o7etriate aboaegse Yo: eleyiaos 
29. {g07HoOD .atnebute opeliod agroiaert A bos AeoTe 
S\TF=TeP gt2 ,0PCt yvpolodaued. dmigoe bas Isazonds < 
in@a » eepaotoMtth feubiv hod + nacerery ban .9el zosxouT 
raet ,etittagodoyed . pai (son ae et ats 7 
| =e 


_omiebiegeind let? Io “enenovisoqite th os? «1.9 ynonze¥ 


= 


) ofc on 


192 


Journal of Personality, 1972, 40, 366-391. 


Wachtel, Peis Field-independence and psychological 
differentiation: reexamination. Perceptual and 


Wagenaar, W.A. and Padmos, P. Quantitative interpretation of 
stress in Kruskal'ts multidimensional scaling 
technique. British Journal of Mathematical and 
Statistical Psychology, 1971, 34, 101-110. 


Wexler, Kenneth M. and Romney, A. Kimball. Individual 


variations in cognitive structures. In 
Multidimensional Scaling: Theory and Applications 
in __the _Behavioural  __ Sciences Volume 2, 


Applications. ed. by R.N. Shepard, A.K.Romney, and 
S.B. Nerlove. New York: Academic Press, 1972. 


Wiggins, Nancy, and Fishbein, Martin. Dimensions of semantic 
Space: a problem of individual differences. [In 
Semantic Differential Technique: a Sourcebook. ed. 
by J.G. Snider and C.F. Osgood. Chicago: Aldine 
Publ., 1963. 


Wish, . and Carroll, J.D. Applications of individuai 
differences scaling to studies of human perception 
and judgment. in Handbook of Perception, Vol If: 
Psychophysical Judgment and Measurement. ed. by 
Ef. Carterette and M.P.Friedman. New York: 
Academic Press, 1974. 


Witkin, H.A. Individual differences in ease of perception of 
embedded figures. Journal of Personality, 1950, 


Witkin, H.A. et _al Psychological Differentiation. New York: 
John Wiley and Sons, 1962. 


1WeRaor gow GRC ete i 


Eso ine lodoye hink \cqakdoaepal-bi0t9  fetive® 


« fsypeasoned «votteoet saottat 
SAE DERRISSS  wAE“ONE ee SCPE 


t% notéatotmreosad avi sathbnaie 2 ,towhet Dae shee sThene pee 
—ntten>  inoofaaog theta _ rete at gern 
she [ugh tee eatin’ 30. Ieee eek Oe snupi ndees 

‘t- -ror gE uty ne 


Lenhivihat .ffedmi¥ <A wana hos «8 ddonnok srelxel 


(7 ye tout t= evitinpoo ai enoissirev 
egnbeua tiga. inp *aopat. spges,soc 25 gu gaih i 
.  omproy — penaeke 


Len ,wegnot.%.8 yh tede 


sEVP? .geast = tandand pixoY wox ne ee «Swe 


Lieonmon to ameinnentt, .aiatet: qateddutt has ,fonae ,aaippie 
; scendare)tr6 Io biv kbot Yo woldow & sanege 
asoodeoguds me per ; 


>? Ta +Onik Sra oopHd | . é Th 19h ne 4 ef. Yd 
-eaet yeldut 


bey thei TO snot* nor Laan oe Gb vl lot%62 hae ~ wdate 
fio ogy enud to peihgse oF podl aoe senna oth 

tip! santsusoaed Jo Aoodbast at ,cvempbot ban” 
4b? .tonspmpe bee | 

~~eo¥ von sateabores.48 


iaydgoduyed 
@¢+o787 369 Per 
sevet seat? Dimoahsoak 


sy) \¢ %) @ana ot fone 7992 if [auhtvibsat .Af wiasiv 
MRE. ak ytiLononzad 20 Lenquet seal? dain 


‘.7oY ve! sobs ol toore VEG Leptpok {eto +88 sobdeew 
a$' noe foe yolt¥ adot 


193 


APPENDIX A 


‘nie ; 
ANT Fone, - ebae 
+54 - ah a rl Bay 


oe |: 
sa ; 
s Y 


194 


CONSTRAINED ASSOCIATION TEST 
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SIMILARITY JUDGMENT TEST 
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SEMANTIC DIFFERENTIAL TEST 
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DEVELOPMENTAL LEVEL TEST 
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COGNITIVE COMPLEXITY TEST 
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UNROTATED LOADINGS OF SUBJECTS ON PRINCIPAL VECTORS 
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UNROTATED LOADINGS OF SUBJECTS ON PRINCIPAL VECTORS 
FOR TUCKER SOLUTIONS - CAU12 
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UNROTATED LOADINGS OF SUBJECTS ON PRINCIPAL VECTORS 
FOR TUCKER SOLUTIONS - CAU12 
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UNROTATED LOADINGS OF SUBJECTS ON PRINCIPAL VECTORS 
FOR TUCKER SOLUTIONS - SD9 
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UNROTATED LOADINGS OF SUBJECTS ON PRINCIPAL VECTORS 
FOR TUCKER SOLUTIONS - SD12 
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TABLE XXX (CONT.) 


UNROTATED LOADINGS OF SUBJECTS ON PRINCIPAL VECTORS 
FOR TUCKER SOLUTIONS - Sp12 


SUBJECT DIMENSION 

ID. 1 2 5 4 
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TABLE XXXI 


VARIMAX ROTATIONS OF LOADINGS OF SUBJFCTS ON PRINCIPAL 
VECTORS FOR TUCKER SOLUTIONS - CAU9 
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TABLE XXXI (CONT.) 


VARIMAX ROTATIONS OF LOADINGS OF SUBJECTS ON PRINCIPAL 
VECTORS FOR TUCKER SOLUTIONS - CAU9 


LS SS A A I SD TE TAD SOS ONS SON RDS OED AOD SOON OEY SE OED canteen eum cen siiin ne SOMO Sam tgs ae Anti vee ee 


SUBJECT DIMENSION 

ID. 1 y 3 4 

44 0.59 ~0.42 0.38 0202 
42 0.98 -0.83 1450 -0.04 
43 0. 03 -0.54 3.26 ~4,.42 
Gy 0.79 0.16 0.26 1.05 
45 -O. 14 =7,56 0.63 1.60 
46 0.11 0.92 1.89 6.5% 
a7 0.03 0.49 a8 2.32 
48 0. 16 0. 73 4.39 0.10 
4g -G.49 -2.%4 3.08 0.87 
50 0.21 -0.74 1.82 0.61 
54 73s 1.43 1.82 2.47 
52 1.46 0.30 0.26 —~0. 10 
53 0.26 0.02 1.68 -0.60 
54 0.52 =.35 0.11 2242 
rs he -0.08 1.42 o.73 0.74 
56 1.37 0.05 2.50 =).25 
57 2.26 —~7.49 0.30 7.09 
58 7.78 =O. 46 0.48 =-0. 06 
59 G.'59 0.93 0.46 -0.02 
60 1.05 0.94 -0.23 -0. 70 
61 Ze 00 0.29 0.44 =. 26 
62 1.13 =Qo57 0.53 0.17 
6 3 0.30 =O. 70 1.292 -0.12 
64 72 =02"0 0.12 0.49 
65 +.65 1.00 0.10 ~O. 41 
66 1.02 0.66 0.10 =O Ps 
67 0.93 A), 02 -G.23 0.48 
68 7. 18 0.43 -0.04 0.07 
69 1.09 0.35 =—).. 02 re 
70 t..33 0.94 ~Oaao 0.13 
71 -§.13 3.32 be be ~LeLe 
72 =O. 94 0.10 1272 0.21 
a 1... 45 0.50 ={) 53 0.66 
74 -. 26 =-9.20 41.29 Gg. 92 
i. 0.49 0.85 C297 0.08 
76 0.82 0.14 ~9. 15 0.17 
VW 6.79 0.08 QO.48 -0.04 


ee ee me ee ee ee ee ee ee ee ee ee ee ee ee ee nw eer eae ee 


=) = ° 

" 7 “ » 

— ee re ae a ee C 
7 =P 


. 2 7. i 
- mt) e7.0 
#6,0- 88,0. 
sf st £0.0 eu ‘ 
Zon re oe 
baat | . é 
ee rn > . 
gt .£ £0.0 va 
Of .0 Ea ee 4 
TP,0 : 
ra = 1820 02 
oF .o- as.f ov .t i ge 
i > ROT a620 £e 
we aS es Ps be 
nv .a 0 = ~ @e 
#“g = 0#,0 Vest ae 
ey) ait Of 40 as.& ve 
oi) A= 90.0 oe.F Re 
60.0- 8.0 2.0 ez 
Tt .0- €$.0+ 20..f oa 
“ O- bad ors ei) 
rr 6 t?.0 Etist sa 
o£.0 £3 

Pn 

va 

: 


225 


TABLE XXXII 


VARIMAX ROTATIONS OF LOADINGS OF SUBUJFCTS ON PRINCIPAL 
VECTORS FOR TUCKER SOLUTIONS - CAU12 
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TABLE XXXII (CONT. ) 


VARIMAX ROTATIONS OF LOADINGS OF SUBJECTS ON PRINCIPAL 
VECTORS FOR TUCKER SOLUTIONS - CAU12 
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TABLE XXXIITI 


VARIMAX ROTATIONS OF LOADINGS OF SUBJECTS ON PRINCIPAL 
VECTORS FOR TUCKER SOLUTIONS - CAS9 
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TABLE XXXIX (CONT. ) 
KRUSKAL SOLUTIONS OF TUCKER SUBGROUP DATA 
CAU12 GROUPS 3,4; CASS GROUPS 1,2,3,4 


DIMENSION 


CONCEPT 1 Z E 1 2 3 
CONC 0.08 -0.20 -0.175 =9.08 -0.05 9.76 
DL SC —0. TF -0. 178 —3. 64 -O.%2 0.07 -G. 94 
EVID —..96 ~9.05 ~-0. 29 “0.05 =¢.73 2.09 
EXPE =0.%% 0.22 0.72 0.08 90.19 0.02 
EXPL ~O0.2% G.07 -0.77 0.06 06.7% -9. 17 
FACT 0.13 0.06 -0.12 0.08 -9.171 -0.04 
HYPO 0.34 -0.06 90.03 ~9. 10 -9. 18. -3.79 
IMAG —~0. 90 ~-G.02 ~¢. 14 -90.01 0.06 -0.00 
INTE 0-10 0.24 0.02 0.07 0.03 0.16 
INVE 0.77 -0.77 0.46 6.22 -0.05 -0.02 
LAW -0.71 -0.00 0.20 9.905 0.09 0.90 
PROO 0.72 8.09 0.23 =9.10 0.70 -0.G63 
PUZZ 0.18 0.05 -0. 174 0.95 -0.07 -0.06 
QUES “$9.78 06.30 -0. 74 =G.99 @G217 -G. 19 
THEO ~0.99 -0.37 0.08 0.07 -0.74 GO. T2 
CONC 0,03 =O. 92 -0..62 =6.08 =9. 97 -0..12 
DISC -0. 03 -6.03 —0. 03 0.46 -G.0F 0.01 
EVID 0.07 =-0.03° (0. 99 G.02 9.22 -0.07 
EXPE 0.03 0.07 0.08 ~Jate —O.03 (9.98 
EXPL 2.08 0.79 -0..03 0.42 =~0.09 0.20 
FACT =. 70 0.03 ©0207 -0.08 -0.2%5 0.07 
HYPO —G.08 0.70 -6.05 G.1% O.75 1.49 
IMAG =§.05 -0.05 -0. 02 ~Q. 72 -0.00 -0.02 
INTE 0.72 =-0.06 -9.01 0.95 —G.22 -0..06 
INVE 0.94 -0.03 -0.10 -~0,02 0.16 ~0. 77 
LAW a.07 G..03.-0.0% =9.07 08.00 0.08 
PROO -0.07 . 0.00 -0. 148 ~0.20. 0.07 -—0..0¢4 
PUZZ 6.08 €.03 -0.03 0.08 ~0.04 -0. 748 
QUES -9.908 9.01 0.11 -0.19 6.73 0.08 
THEO -9.07% -0.710 09.09 9.22 9.08 -0.71 
CONC 9.06 -0.20 90.07 6.07 -8.97 -0.:05 
DAS 9.09 -—0.06 -0.15 -6.06 -0.03 -0.714 
EVID 0.04 0.25 0.04 0.0% -0. 7% 0.08 
EXPE —-0.09 -0.10 0.21 0.78 9.05 -0.00 
EXPL -0.10 0.02 -—0.21 9.08 0.07 0.12 
FACT -0.24 -0.09 0.03 G.03 -0.07 -0..7441 
HY PO 6.23 ~0.13 0.06 —§.0% -0.70 -0. 907 
IMAG 0.04 -0.04 -0.09 6.02 -0.08 -0.04 
INTE 0.09 0.06 0.13 -0.03 6.74 ‘0.06 
INVE -0.06 0.20 -0.12 “8.06 -0.02 0.71 
LAW 0.72 0.02 0.71 G.02 -0.02 4.02 
PROO ~9.07 -O.27T -0. 171 -9. 74 -0..07% 0.00 
PUZZ -0.15 0.06 0.03 0.95 -0.05 -0.08 
QUES G.24 «60.12 -0.09 -0.08 -0.07 -0.04 


THEO -0.14 0.07 0.10 -0.08 0.170 -0.04 
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TABLE XXXIX (CONT.) 
KRUSKAL SOLUTIONS OF TUCKER SUBGROUP DATA 
CAS12 GROUPS 1,2,3,4; SJ9 GROUPS 1,2 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC 0.20 -0.78 ~0..06 9.13 —G.058 0.06 
DISC 0.0% 0.10 =-0.23 0.02 6.16 06.01 
EVID =0.29 0.05. 0.46 —9.40 -G.t? 9.02 
EXPE Gets ~Os22 05,74 0.09 0.01 0.21 
EXPL =—Ge25 0.72 -0, 06 =f.78 “0.06 0.71 
FACT 0.44 =-0.72 -—9.15 0.71 -0.76 0.08 
HYPO -0.27 -90.06 =-6, 23 0.78 -0.07 -0.00 
IMAG 0.01 0.11 -0. 02 0.017 0.04 0.07 
INTE $2.05 9.05 0.27 0.08 0.05 -0. 16 
INVE =“G.21 -0.76 0.09 0.08 -0. 90 -0. 79 
LAW ~0.02 —0527 -0.02 0.70 0.09 0.06 
PROO -0.01 G.23 ~G0. 03 -0.16 06.09 -0.05 
PUZZ -0.01 -0.06 ~-0. 10 G.02 —0006 -0.01 
OUES 0.2% 0.05 -9.03 “GoW Oot? -0. 39 
THEO 0.04 -0.14 0.23 8215 —-0..02 -8.12 
CONC G.07 -G.08 0.02 O22 -91.02 -O.t2Z 
DISC -0.00 -0.08 -0.06 Fat ees -OST3 
EVID “O.0T O.7T@ -9.09 “Geer O42 “Os 02 
EXPE —U4 02 =G. 17 0.56 Get -Gs2is 0572 
EXPL ~0.08 6.04 0.174 “92209 ~0.79 8.08 
FACT ~0.06 -0.09 -~G6. 12 O.05 -9.18 -0.48 
HYPO -~§.15 0.08 -0. 62 =-O.98 7.08 0.26 
IMAG 0.02 6.05 0.02 0.09 0.00 0.15 
INTE 0. OF -0.19 -G.07 G.07  G.27 &..73 
INVE 05.74 -0.03 0..71 —G@.23 =-€.13 -6.04 
LAW -0.1% -0.08 0.05 G.07 -9.09 0.71 
PROO ~9..01 .0.47% =-0. 4 6.14 0.08 90.20 
PUZZ -9.08 0.77 ©. 00 0-77 -0.74 -0.74 
QUES 6.10 0.00 -0.14 Got? 8.26 —-0.13 
THEO 6.73 G.17t G.67 -O.77 6.00 -0.27 
CONC -0.03 -0.20 0. 16 -§,.09 6.94 9.05 
DISC 0.93 -0. 7S —0. 19 0.08 -0.06 -0.98 
EVID =§.09 0.02 =0. 27 O.06 -0.05 ~0.19 
EXPE —9.20 -0.2t 0.90 “9.909 8.10 -0.03 
EXPL -0.13 0.24 -0.14 0.717 0.08 0.096 
FACT O.27 -0.23 0.05 =G.07 ~O.02 -0.06 
HYPO 0.74 -9.04 -0.22 -~§.03 -0..77 -0.02 
IMAG -0.06 -—0.06 -0. 93 0.04 0.06 -0.04 
INTE 0.27 0.00 9.03 =0.03 -0.05 0.07 
INVE Goi? G.22 =-G.09 G506 =0.07 0.01 
LAW -0.24 0.04 90.02 -~0.10 -~-0.04 0.01 
PROO 0.04 0.18 0.20 0.06 0.03 -0.07 
PUZZ -0.05 0.12 0.02 -0.07 -0.07 ~0.04 
QUES 0.74 -0.01 ~-0. 16 0.03 0.04 0.09 
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TABLE XXXIX (CONT. ) 
KRUSKAL SOLUTIONS OF TUCKER SUBGROUP DATA 
SJ9 GROUPS 3,4; SJ12 GROUPS 1,2,3,4 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC 0.05 0.00 -0.15 0.16 -0.01 -0. 14 
i 0.75 -0.08 -0. 04 -05 18 -0..15--0..01 
EVID OG. 1% -0.03 -9.05 9.25 -9.08 %. 05 
EXPE 0.90 ~9.. 42 -O. 14 0.02 -0. 793 -0. 72 
EXPL ~0.08 -0.17 ~0.04 9.03 ~0.736 -0.26 
FACT -0. 08 -0. 30 0. 72 —$.2% 3.00 =%.,13 
HYPO -0.02 0.906 -0. 15 0.07 0.22 0.02 
IMAG 0.06 -0.00 0.02 =O. 96 GO. 0% -0.03 
INTE 6.03 -0.92 -9. 17 0.09 0.02 90.26 
INVE —9.Ti 6.93 =-0.07 =—0.07 06 -.23 
LAW 0.02 -0.73 0.01 0.00 0.17 ~¢.05 
PROO 6.78 -0. 42 -0.62 —“f.At82 °“G.03 -0.19 
PUZZ “Be. O2 -O 90) -O5 44 0.76 0.08 ~-0.905 
QUES 6.04 0.13 0.11 “G.70 0.06 0.18 
THEO —0.75 -0.06 0.09 0.72 -G.19 0.04 
CONC 0.74 -0.79 -0.03 G.06 °@.72 0.02 
DISC -9. 18 -§.08 0.05 “SH. t2 -0.95 ~0..36 
EVID —0.02 6.25 -0.02 0.78 9.09 -—0.906 
EXPE ~-0..02 -6.20 -0. 05 —9,2.18 OG. 42 -0. 11 
EXPL O.42 -O0.55. G. 92 =39 -0.00 -0.05 
FACT =o. tt —Go93 —0. 32 -0.08 0.73 0.99 
HYPO 0.74 -0.00 -0.08 0.08 -0.07 0.12 
IMAG -0.08 -0.07 9.78 =-0.03 0.08 0.07 
INTE -0.06 0.08 -—0.06 -O. 16 ~O.090° 0263 
INVE 0.91 0.05 ~-0.22 =-9.07 -G.05 0.79 
LAW ~9.08 -0.16 6.12 S209 =05 PR. 8.34 
PROO 0.20 0.714 0.04 —-G.~13 -0.78 0.05 
PUZZ 0.03 —-0.602 —0.76 0.04 -0.07 -0.08 
QUES 0.95 0.098 06.12 0.02 G.0F -O.15 
THEO =—Q.12 Gt 0.09 S05 —-G. 73° O.93 
CONC 0.08 -0.09 9.00 9.49% ~G.89 —0..09 
DISC -~0.07 -0.07 0.02 -O.t% 0.08 -0. 91 
EVID -§..02 0.47 -3..04 —§..05 -0.05 -G. 17 
EXPE -9.08 0.03 9.09 =f.02 -0..7S. 0.83 
EXPL 0.05 -9.09% 0.171 -G. t2 —-0. 99 -0.09 
FACT -§..08 -0..63 -0. 07 —0.08 -0.05 0.175 
HYPO 0.09 -0..72 -0..01 Cot? @.02 G. 1 
IMAG -0.00 0.00 -0.01 .06 -0.01 -9. f2 
INTE 0.08 O. 70 -0.01 “G08 0.70 0.72 
INVE 2 -3..66 —0. 01 O2.7F GO. FF -O 1 
LAW -0.01 -0.03 0.08 O.77 -0.06 .-0..07 
PROO 0.06 —0..08 —0.. 10 0.0% G..09 -0. 19 
PUZZ 0.01 0.07 0.03 0.05 ~G.. 07 ~0.51 
QUES 6.00 8.07 -0..09 -0.04 09.70 0.03 
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TABLE XXXIX (CONT. ) 
KRUSKAL SOLUTIONS OF TUCKER SUBGROUP DATA 
SD9 GROUPS 1,2,3,4%; SD12 GROUPS 1,2 


DIMENSION 
CONCEPT 1 2 3 1 - 3 
CONC 0219 -G.08 0.05 S298 —Ge2t CazT 
DISC 0.27 ~G.02 -0. 16 Gets -O.17 <0. 10 
EVID ~0.18 9.09- 0.145 -0.78 -0.04 0.26 
EXPE -%.00 -0.78 =G. 70 -0.08 -0.15 ~0.20 
EXPL G.03 0.70 —6. 22 ~“O2te “Os 0S =0s25 
FACT Gut Ost -G. 18 “95035 “Tse “O.47 
HYPO 0.03 0.05 9.22 0648 0.29 -0.902 
IMAG “0575 O209 ~0.78 -0.035 0.29 ~0.77 
INTE —-0.00 =-0. 99 -0. 177 0.0% -0.34 0.03 
INVE 0.08 0.27 0.01 0.36 G2.06 0.24 
LAW -G.07 -Q. 78 0.72 “0.39 0297 -0.09 
PROO —U.25 -G.87 “0.01 0.22 -0.03 -0.32 
PUZZ G.17 0.97 0.08 0.20 ~0.04 -0.01 
QUES ~9. I2 ~0.20 —9.07 -G.12 0.77 0.798 
THEO -§.02 -0.20 €. 95 -90.24 -0.21 0.06 
CONC $24 —G,.40 =0..02 =0.12 ~92.03: 0.36 
DISC 0.04 0.05 0.19 -0..19° O505 =0.33 
EVID Ott ~OQ.e —bs te io a Ot eee 
EXPE =O. 05! -G526 0.5.63 Geen 8502 -0.32 
EXPL 0.24 -0.12 0.04 0.35 —-0.170 —0.. 13 
FACT =O521 . O248 4 05.12 =9.42 0.02 0.02 
HYPO 06.12 0.00 -0.11 —O200 0224 =—9..18 
IMAG eU.-40. Go42 —Os ie 0.13 6.34 —0.13 
INTE GoU2 =U209 0.2.24 0.793 °-0.497* 0.46 
INVE -~0.903 -0.09 -0..06 -O071o 0298 “O.5.97 
LAW 0.02 0.20 -0.01 -~0.19 =-0.29 0. 74 
PROO GO, ia 0.99 -0.178 9247). 8.23: “0.08 
PUZZ -0.05 0.0% —0. 20 9.00 -0.15 -0).05 
QUES =9.09 0.02 0.42 G.13 0.08 -0.35 
THEO 0.75 G.17% 0.10 0.28 -0.08 0.18 
CONC 60.08 -0.08 0.05 O23 O.03 0. 18 
DISC 0.21 -0.02 9.13 Oo07 ott --0. 06 
EVID 0.08 6.76 9.01 =. 490 —0.05 9.08 
EXPE -9.06 -0.05 0.00 -0.08 0.11 -0.04 
EXPL —0. 01 -0.217 0.21 039017. 0.05: —0..10 
FACT —9.74 0.07 -0.23 =—9.03° O.08 O.42 
HYPO 0.19 0.02 -0.06 0.08 0.00 0.02 
IMAG -0.07 0.06 0.19 0.03 -0.05 —0. 09 
INTE -§. 08 -0. 19 -0.03 =0.08 -—0.07T -0.01 
INVE 0670 —0.416 -0..91 Yot2 -0..07 -0.03 
LAW O.12 0.270 —0. 26 =. 07 -—0.08 -0. 173 
PROO ~0.30 -0.00 -0.02 -Q.G2 “OU. 19> 0201 
PUZZ -9.11 0.03 0.01 0.04 -0.08 0.10 
QUES -0.05 0.19 -0.05 =-0.02° 0.71 8.06 
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TABLE XXXIX (CONT.) 
KRUSKAL SOLUTIONS OF TUCKER SUBGROUP DATA 
SD12 GROUPS 3,4 


DIMENSION 
CONCEPT 1 Z 3 1 2 2 
CONC “Gaus =O.20. 0.2.07 0.22 0.12 ~-0.92 
DISC 0.04 -0.20 -0.08 “0.24 0210 —-0. 16 
EVID CG) 0.75. 9.2:15 “202 —-O.02 =0.04 
EXPE O..70 -0..065- 0.71 0.02 0527 -0.03 
EXPL 30.44 0. 02.--0.-08 “Dele —-OG.06- U.25 
FACT 0.15 0202-0. 05 -0.07 -0.39 -0. 26 
HYPO 0.18 -0.06 -0.06 0.63 0203 =0.26 
IMAG =O.17 Os2ta=-0.07 $2.92 -0sJB48 0.03 
INTE OG. 08 -0.08" 0.15 9371 -02208 0.24% 
INVE -@. 12 -0.09°+-0._08 0.37 -O2G P29 0.905 
LAW 0.03 G248>—-0.03 -0.26 ~O0219' +002 
PRCO —O.. 07 -O -0..20 0.04 0.00 0.19 
PUZZ 2. 08 - 8208 7-7.-743 0.16 -JU20e 0. 78 
ODES -0.174 -0.02 0.08 “0.22 O.2% 96.08 
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TABLE XL 


LOADINGS OF SUBJECTS ON DIMENSIONS FOR CARROLL 
SCLUTICNS CF FULL GROUPS —- CAUS 


ae ee Ne ee ee ee ee ne ee ee 


SUBJECT DIMENSION 
ID. 1 2 3 
1 0.33 0.16 0.27 
2 0.18 0.45 0.49 
3 0.53 0.22 0.26 
4 0.15 0.26 0.18 
5 0.29 0.36 0.23 
6 0.24 0.38 0.49 
7 0.22 0.40 0.42 
8 0. 30 0.26 0.66 
9 0.18 0.42 0.48 
10 0.47 0.33 0.23 
11 0.45 0.29 0.33 
12 0.43 0.15 0.45 
13 0.35 0.50 0.26 
14 0.33 0.25 0.27 
15 0.39 0.35 0.33 
16 0.27 0.33 G47 
17 0.18 0.61 0.26 
18 0.26 o.22 0.30 
19 0.35 0.31 0.32 
20 0.18 0.36 0.16 
21 0.23 0.38 0.22 
22 0.33 0.17 0.25 
Z 0.20 0.21 0. 30 
24 6.34 0.39 0.24 
25 0.47 0.24 0.25 
26 0.36 0.25 0.24 
27 0.19 0.42 0.31 
28 0.27 0.22 0.28 
29 0.38 0.34 0.28 
30 0.28 0.36 0.24 
31 0.38 0.31 0.12 
32 0.28 0.40 0.37 
33 0. 26 0.36 0.47 
34 0.40 0.34 0.37 
35 0.47 0.37 0.20 
36 0.46 0.35 @..31 
37 0.14 0. 36 0.51 
38 0.87 0.26 0.31 
39 0.23 6.27 0.20 
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TABLE XL (CONT.) 


LOADINGS OF SUBJECTS ON DIMENSIONS FOR CARROLL 
SOLUTICNS OF FULL GROUPS - CAU9 


SL SS a a a ee ee 


SUBJECT DIMENSION 

ID. 1 2 3 

44 0.49 0.31 0.22 
42 0.48 0.41 0.38 
43 0.22 0.23 0.57 
uy 0.39 0.49 0.38 
45 0.28 0.31 0.37 
46 0.41 0.29 0.39 
47 0.24 0.20 0.25 
48 0.24 0.37 0.35 
49 0.27 0. 14 0.67 
50 0.42 0.15 0.38 
Se 0. 04 0.28 0.19 
52 0.45 0.35 0.36 
53 0.31 0.29 eT 
54 0.18 0.66 GO. 77 
se 0.35 0.28 0.40 
56 0.37 0.38 0.23 
92 0.36 0.46 0.33 
58 0.46 0.29 0.35 
a3 0.34 0.39 0.25 
60 0.36 0.42 0.23 
61 0.47 0.40 0.32 
62 0.47 0.37 0.45 
63 0.22 0.30 0.56 
64 0.51 0.32 0.32 
65 0244 0.36 0.28 
66 0.40 0.38 0.23 
o7 0.35 0.21 0.33 
68 0.41 0.34 0.24 
69 0.39 0.35 0.19 
70 0.32 0.32 0.26 
77 0.22 0.21 0.45 
72 0.29 0.20 0.22 
73 0.34 0.39 0.16 
74 0.27 0.37 0.27 
75 0.39 0.38 05.3% 
76 0.33 0.24 0. 20 
ef 0.34 0.34 0.40 
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TABLE XLI 


LOADINGS OF SUBJECTS ON DIMENSIONS FOR CARROLL 
SCLUTICNS OF FULL GROUPS - CAU12 


LLL AS SS AG ome ce ee ee 


SUBJECT DIMENSION 
£0. 1 2 3 
1 0.27 0.25 0.30 
2 0.30 0.30 0.34 
3 0.28 0.24 0.24 
u 0.41 0.38 0.22 
5 0.22 0.16 0.29 
6 0.24 0.37 0.28 
7 0.40 0.25 0.31 
8 0.28 0.35 0.49 
g 0.43 0.33 0.32 
10 0.29 0.28 0.26 
11 0.55 0.41 0.15 
12 0.29 0.55 0.23 
13 0.35 0.34 0.42 
14 0.33 0.16 0.44 
15 0.26 0.30 0. 30 
16 0.29 0.38 0.20 
17 0.48 0.43 0.30 
18 0.43 0.39 0.25 
19 0.10 0.17 0.68 
20 faa 0.30 0.34 
21 0.23 0.41 0.14 
22 0.39 0.36 0.31 
23 0.35 0.27 0.17 
24 0.56 0.26 0.37 
25 0. 38 0.30 0.36 
26 0.43 0.42 0.24 
27 0.54 0.31 0.15 
28 0.30 0.23 ea 
29 0.38 0.28 0.40 
30 0.38 0.13 0.26 
31 0.38 0.46 0.19 
32 0.20 0.25 0.25 
33 0.29 0.19 0.18 
34 0.49 6.27 0.36 
35 0.23 0.39 0.13 
36 0.41 0.35 0.30 
37 0.19 0.25 0.174 
38 0.41 0.19 0.792 
39 0.57 0.28 0.25 
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TABLE XLI (CONT.) 


LOADINGS OF SUBJECTS ON DIMENSIONS FOR CARROLL 
SOLUTICNS OF FULL GROUPS -— CaU12 


— em een ae a eae ea ae a a ee 


SUBJECT DIMENSION 

ID. 1 2 3 

41 0.39 0.36 0.20 
42 0.42 0.36 0.26 
43 0.34 0. 20 0.30 
44 0.35 0.42 0.22 
45 0.33 G32 0.39 
46 6.45 0.29 0.34 
47] 0.21 0.52 0.34 
48 0.30 0.45 0.30 
4g 0.54 0.37 0.28 
50 0.27 0.28 0.41 
51 0.35 0.38 0. 45 
22 0.33 0.25 G.42 
a3 0.41 0.26 0.25 
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TABLE XLII 


LOADINGS OF SUBJECTS ON DIMENSIONS FOR CARROLL 
SOLUTIONS OF FULL GROUPS - CAS9 


—_——— em me rm re rn ee ee ee eee eee ee eee 


SUBJECT DIMENSION 
ID. 1 2 3 
q 0.27 0.34 0.16 
2 0.49 0.19 0. 44 
3 0.27 0.53 0.21 
4 0.17 0.16 0.26 
i) 0.23 0.30 0.36 
6 0.49 0.25 0.36 
7 0.43 0.22 0.39 
8 0.67 0.29 0.26 
] 0.50 0.17 0.41 
10 0.24 0.48 0. 33 
11 0.34 0.45 0.28 
i: 0.48 0.41 0.14 
13 0.27 0.35 0.49 
14 0.28 0.32 0.25 
15 0.34 0.38 0.35 
16 0.17 0.27 0.33 
17 O.27 0.18 0.61 
18 0.31 0.26 0.23 
19 0.32 0.35 0.31 
20 O..17 0. 18 0.35 
21 0.22 0.24 0.37 
22 0.25 0.33 0.77 
23 0.29 0.21 0.21 
24 0.24 0.21 0.39 
2 0.27 0.47 0.23 
26 0.25 0.36 0.24 
27 0.31 a... 49 0.42 
28 0.28 0.27 0.22 
23 0.29 0.37 0.33 
30 0.25 0.28 0. 36 
31 0.12 0.40 0.30 
32 0.38 0.27 0.40 
a2 0.47 0.27 0.35 
34 0.38 0.40 0.34 
a 0.21 0.48 0.36 
36 0.32 0.45 0.35 
37 0.51 0.14 0.36 
38 0.33 0.46 0.25 
49 0.20 0.24 0.26 
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TABLE XLII (CONT. ) 


LOADINGS OF SUBJECTS ON DIMENSIONS FOR CARROLL 
SOLUTICNS OF FULL GROUPS - CAS9 


ee ee ee em ree me ee ne ee nee ee ee ee ee ee ae 


SUBJECT DIMENSION 

ID. 1 2 3 

41 0.23 0.50 0.31 
42 0.38 0.49 0. 40 
43 6.57 0.21 0.23 
44 0.40 0.38 0.49 
45 0.37 0.28 0.31 
46 0.40 0.38 0.30 
47 0.25 0.25 0.19 
48 0.36 0.23 0.36 
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TABLE XLVIII 
KRUSKAL SOLUTIONS OF CARROLL SUBGROUPS 
CAUS GRPS 1,2,3; CAU12 ERPS 1,2,3 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC Gantt ~Gs TP =O. TH C205 =-G..04 -8.02 
DISC =G. 712 G.t9 °0.09 0.02 0.71 -0.98 
EVID 0.04 -0.92 -0..22 iis -OC.04 “Fs i3 
EXPE —-G.25 -0.06 0.10 “6594 =0.7% =-0.15 
EXPL “0219 G.2.0F =@. 1% -~0.06 0.02 =-0. 76 
FACT 0.02 -0.16 0.18 “9.05 -0. 16 09.09 
HYPO 0.39 G2 72 —6. 12 “Get? -G.09 =3.92 
IMAG -0.03 6.605 0.71 0.02 0.71 “8.35 
INTE 6.713 O.77 0.16 6.75 ~G.08% 0.03 
INVE 9.20 -0.06 6.17 7.02 9296 9.77 
LAW “0.02 0.19 0.04 -0.77 -0.0T 0.70 
PROO —=Gs47 ~O. 95 -8203 “0.08 G.37 0.06 
PU ZZ 0.16 -G.7? ~C.05 O.0F —0..70 0.72 
QUES —0500° 70278-0219 -0.16 0.08 -—0.05 
THEO —-0.05 -0.21 0.02 $2.15 0.7% 0205 
CONC =0.09 9.98 0.99 G..499 ~-0.02 =G.:09 
DISC —§0207 -0.73 -0.1V6 “0.79 ©.06 -0. 72 
EVID -0. 12 ~9. 71% -0.08 GCo22 -0211 0.15 
EXPE GO. G2 -0.03 —0. 173 0.905 0.25 -0.06 
EXPL G2tt 8290 —0..92 =0.12 0.20 6.92 
FACT 0.08 -0.18 -0. 01 6.04 0.08 -0.25 
HY PO =—95. 07 -0.495  -G.205 S03 OO. 1% -O.23 
IMAG -0.33 6.04 90.05 -0.13 -90.05 0.14 
INTE G..76 —0.03 —6. 06 G.17 OG2.10 O21 10 
INVE 0.76 -G.07 0.12 =—§.91 =-0.23 0.10 
LAW 6.95 0.717 0.02 “0.05 -G.13 0.12 
PROO ~§.12 ~0.70 0.10 “9.23 0.07 0.174 
PUZZ 0.08 0.13 0.08 02178 “0.02 -O.13 
QUES ~0.274 °0.:03 —0. 73 0205 =-0e22 "0215 
THRO 6.02 -0.07 0.17 “0.175 -@d.13 ~-0. 17 
CONC 0.02 0.78 0.76 0.04 0.06 9.25 
DISC —-0.23 -9. 06 -@.745 0.08 0.76 -O.22 
EVID -0.06 0.41 0.07 <9 .2% —0.08 -0. 15 
EXPE -O.27 0.96 0.17 -0.21 -0.14 0.16 
EXPL —-0. 78 6.72 =-0.29 -0.18 0.08 0.11 
FACT -0.27 -0.21 0.10 0.22 -0.06 0.16 
HYPO 0.07 ~0.07 -0..38 0.19 =-0.00 -Os 14 
IMAG Oo? 8.4 =—0.909 0.06 ~0.05 0.16 
INTE 0.17 -0.10 0.31 —=§207 -0.15 -O.25 
INVE 0.16 —Os32 ~0..02 0.77% -02.146 =0.05 
LAW -0.07 -0.30 -0.07 -0.03 -0.24 0.04 
PROO 0.327 -0.70 -—0. 10 =-9,10 0.19 -0. 72 
PUZZ 0.23 -9.77 G.05 -§.16 0.03 -0.07 
QUES 0208 eGs 15-02 75 0.19 0.12 0.04 
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TABLE XLVIITI (CONT. ) 
KRUSKAL SOLUTIONS OF CARROLL SUBGROUPS 
CASS GRPS 1,253) “GRSI246RPS)152 93 


DIMENSION 
CONCEPT 1 Z 3 1 2 3 
CONC “0,08 -O. 21) ~0.06 -0,06 <0. 74 —9.19 
DISC —~§. 43 -0..07 -0.25 -0.02 0.13 -0. 78 
EVID 0.27 0,05 -0. 24 0.95 -0.04 -0.24 
EXPE G.10 -G.21 ~0. 17 “0.75 ~-0.20 -€.09 
EXPL 0.28 -G.07 9.02 0.07 ~@.22 -0.67 
FACT 0.96 -0.15 0.24 =6.26 0.912 -—0.06 
HYPO -0.21 -0.08 -0.06 G.ar ~9.03 ~0..02 
IMAG 0.02 -0.78 0.18 “0.43% -0..42 0.174 
INTE 0.10 0.77 0.20 0.20 0.173 0.05 
INVE =9.906 O.26 «0.12 H.02 0.8% -O.17% 
LAW “0.76 6.603 0.20 =0.05 0.78 0.72 
PROO ~0.30 -9. 76 -—0. 06 G.70 -G@.73. -G.21 
PUZZ 0.29 0.20 0.03 0.00 0.78 -0.11 
QUES =O. 73 ~0.260 <9. 19 0.03 -0. 74 -0.16 
THEO C201 G.20 =-0. 75 =—OQ.28 —-0.07 —-O.07 
CONC o.97 =-0.82 0.72 -0.13 -0.04 0.01 
DISC 0.0% -0.908 —0. 06 0.06 0.01 -0. 16 
EVID -9.07 -0.74 0.08 S.00 -G. 75 -0. 93 
EXPE -0.06 0.12 0.09 =0.503 -0.20 =-0.91 
EXPL 0.83 -—@.. 18 -0.08 =).05 -G.12 0.74 
FACT —G.7T7 -0.02 -0.:03 -90.14 0.74 -0.05 
HYPO 0.70 0.170 6.06 0.18 9.13 -0.02 
IMAG 9.02 0.02 -0.05 GO. 47 -3.00 0.73 
INTE -0.07 -0.07 -0.04 0.03 0.74 =-9.12 
INVE “le DF —-9.03 —G..45 ~“O.017 0.78 0.06 
LAW -$.05 0.43 0.91 GS. 29 =8. 07. -0..07 
PROO 0.75 -@.99 -0.¢7 G.07 0.05 0.77 
PUZZ -9, 05 =—0.02 9. 13 O0.76 -0.04 -G6.09 
QUES -0.08% 0.12 -G. 11 =0.17 8.08 -0.173 
THEO 0. 14 -0.08 0.05 =~7. 16 -G..05 -02 73 
CONC 0.719 0.07 0.19 -~O..08 - 9.06 0. 76 
DISC -O.21% -9.09 -0. 09 -Q.40 <G.200 -0..45 
EVID 0.29 0.16 -0.04 Ot) -O. 19 -0..7F7 
EXPE -0.07 0.21 90.23 -9.00 0.23 0.03 
EXPL “9.79 0.20 —0. 22 ~0.06 0.74 -0.71 
FACT -0.29 0.08 0.16 0.07 ~0.72 0.78 
HYPO 0.03 0.07 -0.34 -§..09 -@. 78 -0.11 
IMAG 9.09 @.77 0.04 -0.04 0.03 0.14 
INTE 0.78 ~8..27 9.74 60.79 -0.93 -0.03 
INVE 6.01 ~0.30 -06. 15 -0.00 -0.20 -0.04 
LAW ~§. 23 -0.12 -0.07 =. 92 -9..05 -0..01 
PROO 6.73 ~O. 94 -0.22 0.72 G.49 -G.0f 
PUZZ 0.04 -0.17 0.14 0.9% 9.05 +0292 
QUES 0.02 -O.27 -0.02 -0.16 -0.08 0.07 


THEO -0.04 -0.18 0.25 0.16 0.08 0.11 
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TABLE XLVIII (CONT.) 
KRUSKAL SOLUTIONS OF CARROLL SUBGROUPS 
SJ9 GRPS 1,2,33; S312 GRPS 1,2,3 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC -0.05 -0.13 -0.20 ~0.172 -0.056 0.18 
DISC 0.13 -0.23 0.02 -0.03 -0.07 -0.20 
EVID 0.18 0.22 -0.04 0.00 -0.21 0.174 
EXPE 0.2% -0.04 -0.12 -0.21 -0.10 -0.05 
EXPL 0.10 0.15 0.25 0.09 -0.19 -0.07 
FACT 0.09 0,69 -6.27 -0.01 0.18 0.01 
HYPO -0.16 -0.08 0.11 =0.1714 0.174 -0.07 
IMAG 0.18 -0.02 0.06 -0.00 -0.03 0.13 
INTE 0.09 -0.10 0.22 0.22 -0.02 0.01 
INVE -0.27 0.06 -0.08 O12 9.092 9.20 
LAW -0.16 0.07 0.19 ~0.13 -0.02 -0.09 
PROO -0.18 -0.26 0.06 0.71 0.117 -0.14 
PUZZ -0.03 0.21 0.07 6.02 -0.0% -—0.17 
QURS —0.0% -0.14 -0.17 0.76 0.12 -0.02 
THEO “0.11 0.21 -0..71 =0,.07 0.17 0.13 
CONC -0.11 -0.07 0.13 -0.06 -0.16 0.16 
DISC 0.16 0.05 -0.06 -0.22 -0.05 -0.14 
EVID -0.04 0.19 -0.06 0.09 0.25 0.09 
EXPE -0.03 -0.16 0.03 ~0.171 0.09 0.25 
EXPL 0.08 0.10 0.16 -0.19 09.18 90.00 
FACT -~0.06 -~0.10 -0.13 0.19 -0.06 0.06 
HY PO 0.07 -0.11 -0.06 0.02 -0.22 0.03 
IMAG —0.03 -9.02 €.07 0.16 0.06 0.08 
INTE 0.03 0.03 -0.14 -~0.25 -0.05 0.07 
INVE -9.09 0.07 -0.14 -0.05 0.04 -0.29 
LAW -0.14 -0.05 -0.04 0.16 -0.00 -0.15 
PROO 0.41 -0.17% 0.12 -0.01 -0.23 -0.12 
PUZZ 0.12 0.01 -0.03 6.0% -0.09 =6.72 
QURS -0.10 0.08 0.09 -0.02 0.23 -0.12 
THEO 0.03 0.11 0.05 6.12 0.00 0.28 
CONC «0,02 =0.97 0.03 0.06 -0.20 0.02 
pIsc =§.27% «0.03. 0.91 0.03 -0.05 0.16 
EVID 0.09 0.06 -0. 11 -0.08 0.18 -0.10 
EXPE -0.02 0.06 0.18 -0.18 -0.05 0.09 
EXPL -0.08 0.06 -0.08 -0.09 0.13 0.06 
FACT 0.74 -0.08 -9.07 -0.07 -0.15 -0..06 
HYPO -0.09 -0.06 -0.03 0.06 0.79 0.75 
IMAG -0.00 0.05 0.02 -<§.02 0.07 -0.01 
INTE 0.02 -0.03 0.06 —~0. 14 =~0.00 -—0.07 
INVE 0.03 0.13 0.02 -0.05 0.02 -0.21 
LAW -9.08 -0..03 9.09 -0.06'-0.06 0.10 
PROO -0.05 -0.07 -0.11 0.18 0.13 -0.02 
PUZZ 0.03 -0.03 -0.08 0.04 0.01 -0.07 
QUES 0.06 -0.07 0.01 0.12 -0.04 0.06 


THEO 0.10 0.06 -0.02 0.75 ~0.06 -0.12 
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TABLE XLVIII (CONT.) 
KRUSKAL SOLUTIONS OF CARROLL SUBGROUPS 
SD9 GRPS 17,2433) SD12 GRPS 1,2,3 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC “0204 —O327 02.46 0279 G20) 9225 
DISC 0.10 —-§. 09.90.24 0.14 -0.10 0.01 
EVID 0.906 -9.30 0..00 ~O..20 -0.07 6.05 
EXPE 0.23 0.01 -0.07 0.70 -0.05 -0. 74 
EXPL 0.12 -0.29 -0.05 -9.09 -0.74 -0.21 
FACT -0.26 -0,77% 0.06 “0.12 0.78 0.70 
HYPO “0.13 .0. 48 -0.03 “0.08 0.22 0.03 
IMAG 0.17 0.19 -0. 03 0.00 Q.05 -—0.,22 
INTE 0.74 =-0.05 -0. 25 -0.05 -0.24 -0.07 
INVE =§0.29 0.74 «0.93 0.92 ~G..72 0.97 
LAW —“Q. 43 -0.22 -0. 128 -0.21 0.02 -0.08 
PROO -0.25 0.07 -0.15 0.3% 0.02 -9.03 
PUZZ 0.03 0.11 -0.24 0.05 8.23 -9..73 
QUES 0.25 -0.05 09.74 0.08 -0.02 0.18 
THEO -O.02 +0.07 -G. 37 0.205 =0.22 0.73 
CONC =0.20 ~0.18 <0.97 Rete Wea —Co07 
DISC 6.02 0.18--0. 174% 0420G~ -8242--0215 
EVID 0.03 -0.06 -0.24 -90.07 0.048 -—6.29 
EXPE 0.02 -0.179 =0..01 0.78 0.93 -0. 16 
EXPL O.22 -0.72 6.00 “0.32 -0.18 0.03 
FACT 9.7910 -0.07 0.22 =0.49 0.25 58.36 
HYPO 0.06. ~0. 43 «0.173 0576 0.07 0.13 
IMAG -0.78 023 -0.743 0.07 -0.28 9.01 
INTE 0.36 0.949 -0.99 0.25 -0.70 2.02 
INVE 0.45 0.07% -0.08 =6..35 =0.07 —0.19 
LAW =“. 39 ©. 38 —0.07 “220 ~—9.92 -06 75 
PROO 0.09 -0.73 0.22 0.05 -0.10 0.26 
PUZZ 0.04 -0.08 -0. 10 9.093 0. 10 —0.97 
QUES -§0.713 08.2079 0.07 Q.00 0.12 0.13 
THEO -9.05 0.97 0.06 =.22 3.07 0.02 
CONC —-9,34 0.07 0.26 =. 03 -0. 12 -G.05 
DISC ~9.22 -0.28 0.177 =—O548. 02460-0432 
EVID 0.07 -0.36 -0.20 =0.73 1.27 0.27 
EXPE —~0. 24 -0. 607 -0.27 G.08 -G..12 -G.25 
EXPL =-G.39  Q.77 =. 06 “G02 ~G.85> O.. 41 
FACT 0.2% -0. 79 —@..19 O.2F -G.87 -0.05 
HYPO 0.09 -0.27% ~9.039 0.06 -0.00 90.31 
IMAG 6.25 @.37 -0. 40 o.22 —-0. 1 -0. 26 
INTE -0.172 -0.77 -0. 19 0.34 -0.09 -0.07 
INVE 9.11 0.10 -0.40 G43 -f.27 -G.92 
LAW 0.01% 0.35 0.24 —O.92 0.06 =—0..77 
PROO G6. 72 -0.06 0.32 0,32 -0..39 -0.26 
PUZZ 0.35 0.08 0.05 0,22 0.73 0.20 
QUES -0.10 0.34 -0.14 —O. 38  O.21 0.92 
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TABLE XLIX 
CARROLL SOLUTICNS OF CARROLL SUBGROUPS 
CAUS GRPS 1,2,3; CAU12 GRPS 1,2,3 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC 9.05 -9.83. 9.05 “0.77 Gs3% 0.28 
DISC -0.75 -0.7% 0.4.36 O.28 G.20 2139 
EVID 0.40 0.28 9.10 G35 Gaz? -O.0F 
EXPE -§. 15 -0.28 0.09 0.15 -G.2P O.32 
EXPL -0.10 -0.30 0.68 =“G.32 Ge2dF 9.94 
FACT 0.40 0.20 0.05 0.28 0.30 -0.T1 
HYPO —§.23 —Gs0" =O. 92 =-0.26 -0.37 -0.15 
IMAG -0. 16 0.24 -0.43 ~Q.29 -0. 10 -8.37 
INTE —0.08 —-G537 -0. 16 -O. 18 -0.23 -0..42 
INVE -0. 01 -0.79 0.26 0.25 -0.32 90.38 
LAW 0.33 @.22 0.09 0.26 .16 -0.25 
PRCO 0.46 9.25 6.09 0.34 0.32 -0.03 
PUZZ -0.317 0.45 6.290 =—O.24 -0.26 0.77 
QUES -~0.32 0.08 6.30 =—“O229 —9. Tt~ 8528 
THEO =9. 746 =O. 13 -0. 34 =G2ir ~U. 25 ~0.32 
CONC G.15 0.20 -0.72 6.30 —0. 223 -0. 17 
DISC -0.22 0.39 -9. 22 0.04 -0.30 0.01 
EVID 0.29 0.32 0.01 0.28 0.47 0.06 
EXPE -§.2% -O8.907% 0.36 0.00 -O.37 0.30 
EXPL 0.28 0.07 -0.14 9.G2 -G. 170 -0.485 
FACT 0.28 0.33 9.05 0.37 6.490 6.01 
HYPO —0. 18 —0.238 =—0. 27 =-9,24 -@. 76 6.75 
IMAG =-9.0% -0.00 -0..34 ~§.36 9.74 -0..149 
INTE Tk -f.32 -Gart ~§. 78 -0..0T -0.50 
INVE rOs.23 -9.78 9.81 =-§.00 -G.34 0.37 
LAW 9.25 9.75 0.0% O.2% O77 -0.05 
PROO 0.33.--G.25 0.08 9.37 9.80 9.10 
PU ZZ —~9.37 -0.28 9.32 “0.33 0.77% “0.31 
QUES “Oo42 -0. 20" 05 38 “Gage 02 OF. 27 
THEO 0.06 ~-9.33 -0.339 =“O.31 =U. 06 -0.22 
CONC -0.13 -0.28 90.903 238 =0.73 —0.35 
BESC “0.24 +6.07 0. 15 O.93 -0.20 -0. 77 
EVID 0.39  €5978 Os 56 G.83 ~GatF <0. 15 
EXPE 0.95 -0.08 ~-0.37 0.02 -0.29 -0.41 
EXPL -0.06 -0.31 -0.09 G08 -O. 085 -0. 028 
FACT 0.392 “74 O.% 6.33 0.74 90.29 
HYPO 0.97 =G.239 -04.26 -Q.28 -0. 26 -0. 12 
IMAG “9.3% 0.07% 0.27 -0.46 0.07 9.19 
INTE -0.23 -0.27 -0.04 —§. 177 0.32 -0. 13 
INVE Go2% G37 -0239 G09 =9.30 ~0. 75 
LAW Ga27 “Gets -O.G2 Gut? -~G. 02 ~9.39 
PROO 0.40 0.16 0.38 0.38 -0.T? ~0.24 
PUZZ -0.39 0.35 0.01 =9,.39 °“D.28 -0.06 
QUES -0.24 0.31 -0.19 —Oe20° -0540->-0.17 


THEO Bhi Osa he. aS -0.19 -0.21 0.48 
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TABLE XLIX (CONT.) 
CARROLL SOLUTIONS OF CARROLL SUBGROUPS 
CAS® GBPS 1,232,322 CAS 12-6RPS 12223 


DIMENSION 
CONCEPT 1 2 3 1 Z 3 
CONC D222 G.at O41 0.05 -0.43 0.04 
DISc -0.32 0.14 0.482 -0..15 -0.74 -0.30 
EVID 0.26 0.22 0.351 0.40 0.28 0.10 
EXPE ~0,29 ~G.28 «-0,35 ~0.45 ~0,.24% ~-0.09 
EXPL 0.08 0.3% 0.07 -~§. 10 -0.39 0.08 
FACT 0.39. 0.99 0.27 0.40 0.20 0.05 
HYPO -0.22 0.04 -0.41 -G.23 0.00 -0. 82 
IMAG -§.07 -9. 1713 ~-~0. 39 =-0.16 0.2% -9. 83 
INTE =—=G.2t G.235 -0, 449 “0.04 =-G.277 -0. 16 
INVE —0.25 ~—0.35 —-0. 08 ~0.00 -0.29 0.25 
LAW 0.42 0.08 0.00 0.32 90.22 0.04 
PROO 2.2% O72 @.27 0.46 0.26 0.09 
PUZZ —-G. 45 -0.86 —0.73 “G.50 G85 0.20 
QUES —~0.15 -0. 57 -9. 72 -§.32 0.96 0.30 
THEO ~0.12 0.27 -0.49 =O. to —G. 14 ~0.58 
CONC ~$.71 O.38 0.28 0.39 -0.23 <0. 77 
DISC 0.24 0.20 0.43 0.08 -0.30 0.01 
EVID 2.325 0.27 -0. 09 0.28 0.87 0.06 
EXPE Go. te -G.27 0.32 0.00 -0.37 0.30 
EXPL —He33. Gish B97 o. 02 —-0..16 -0.485 
FACT 6.28 6.30 -9. 71 0.37 0.40 90.01 
HYPO =%.2429 -6.237 ~9..15 =G.27 —O. to 2.15 
IMAG ~G. 20 0.90 -O2 27 =9.36  @.7t =-0. 18 
INTE =H. 08 —O. 2a —O 82 =O.18 -O. 07 -G.250 
INVE 0.25 -0.32 0.39 -0.00 -0.34 0.37 
LAW 0.26 0.76 -—9..25 Q.27 0.47 -%.05 
PROO 9.34 . 9.32 -0..05 0.37 0.40 0.10 
PUZZ =9. 24 -G.26 -0.. 77 =G,438 “Gath Dost 
QUES —-§.25 -Q. 7? -O..28 =].25 -G.07 2.27 
THEO ~0. 7% =-3.25. -0. 22 “G34 —-0..08 -G..22 
CONC 60.05 0.06 -0.47 0. 18 <0. 75 —t. 38 
DISC -—O.27 GF =-0. 06 G.03 =-0.20 -0..7% 
EVID 0.44 0.26 0.18 G.35 -0. 25 -0. 19 
EXPE ~0.34 0.23 0.04 6.202 =G. 29 +0.483 
EX PL —§. G& ~0.00 -~0. 53 Q.07 9.85 -0.08 
FACT 08% 0.25 0.12 0.33 0.7% 0.29 
HYPO ~9. 9S -0.27 -0.0F =-§.2% -0. 29 -9. 72 
IMAG S07 -0.4983 0.25 =G.96 0.07 0.79 
INTE -§.0T -0.36 -—0.37 —O.1% 0.328 -0.73 
INVE -§.30 0.37% 0. 12 S209 -9.29 -0. 17% 
LAW 0.22 G06 0.06 0.79 -G.03 0.39 
PROO 9.92 Q.2% 0.23 a.28 =O. tt ~O..25 
PUZZ ~0.78 -0.03 6.32 -~G.49 0.28 =0.05 
QUES -0.29 0.02 0.24 —~0.20 0.40 -0.17 
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TABLE XLIX (CONT.) 
CARROLL SOLUTIONS OF CARROLL SUBGROUPS 
SJ9 GRPS 1,2,3; SJ12 GRPS 1,2,3 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC “O.22 OGa2t6 0. 76 Os20 0.43 ~0.98 
DLSE “G79 —G. I0 “8225 0.25 —G..3F ~05 72 
EVID —Oeat® O@699 -0.,352 G.27% 0. FF -0.08 
EXPE 0.44 0.01 -0.07 C.09 =-0.79 -0.35 
EXPL ~O0. 18 @G.28 0.22 0.72 -9.00  0..22 
FACT —G.78 Gs2Z5 C227 O.3% 0.34 °O. 92 
HY PO 0.05 -0.41 -0.30 “90.26 -G.37 G. 12 
IMAG =. 12 =0.49 -0., 39 -G.39 “0.39 218 
INTE -0.77 0.32 -0.45 “et “Coss 0.39 
INVE 0.32 0.05 -0.09 0.74 -0.09 -0.39 
LAW -0.05 0.28 9.28 0.92 G25? OsZZ 
PROO -9..79 @.78 0.33 0.33 0.576 0.06 
PUZZ 0.44 -0.27 -0.14 =G.30 “8.09 -0.35 
QUES 0.42 -0.24 -0.26 -O.3% G20 —0. 46 
THEO +9. 30 ~-G.19 -0.799 -0. 30 -0.16 0.20 
CONC 0.28 0.30 0.77 G.27 8.78 -0.21 
DISC 0.04 0.26 90.41 G.25-0520 0.23 
EVID 046399 On97F <G.08 0.29 9.29 =o. 42 
EXPE ~0,.30 0.36 =-0.07 =G.02 —0..71 0.39 
EXPL 0.18 0.01 0.22 O.2 €.30 -0. 15 
FACT 0.18 0.28 9.11 8.28 0.27 -0.23 
HYPO ~9. 19 -0.34% -9.28 —Goa0 “UV. 22 —0..05 
IMAG +~0,591 -G.56 0.25 -§.37 -0.48 -0. 14 
INTE G. 76. -G. 19. 0.25 Oo. 79 -3. 20 -O. 348 
INVE -0,08 9.78 —0. 07 “0.07 -0.05 0.47 
LAW 0.33 -0.20 -0.28 0.05 0.85 -0.25 
PROO 6.23. 0.22 0.78 0.25 0.25 -0. 36 
PUZZ =“9.52 ~0.07 -0.38 —-§.38 -0.23 3.398 
QUES “OG, 27 ~O.02 —-0. 82 “3.89 -G.900 «0.35 
THEO G. 10 -9.33. -0.31 —-0538 -0.70 —0. 16 
CONC 0.716 0.77 -0.06 0.18 0.18 -0.24 
DISC 9.05 0.38 9.29 9.25 -0. 77 -9.04 
EVID 0.24 0.19 -0.21 6.29 0.05 -0.01 
EXPE 0.08 0.05 0.26 —G.0F —G.03 -O. 72 
EXPL G09 ~0.43 -9.99 $08 6.793 -0.25 
FACT 0.34 0.09 -0. 22 G38 ©. 73 ~0..05 
HY PO -0.35 0.09 0.07 =. 7% O.t% -0..37 
IMAG —§,. 53 -0.53 -0..05 -0.45 -0.59 0.28 
INTE -§.32 0.00 -0.27 -0.24 0.06 -0.30 
INVE 0.24 0.08 0.25 “O00 OQ.:29 @. 77 
LAW 0.27 0.06 -0.4%6 0.32 0.08 0.28 
PROO 0.26 0.24 -—0.08 6.30 G.04 -0.03 
PUZZ -9. 27 -0535 0.52 ~§.38 =-0.6% G37 
QUES 6.07 -0.47 9.25 -0.33 0.25 -0.49 


THRO ~0.37 -0.20 ~0.79 ~0.05 ~0.24 =0.33 
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TABLE XLIX (CONT.) 
CARROLL SOLUTIONS OF CARROLL SUBGROUPS 
SD9 GRPS 1,2,3: SD12 GRPS 1,2,3 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC -O220 0203 0222 0208 0279 -Oi27 
DISC 0536 -0534°-0227 0.23 —0.17148 -0.53 
EVID 6.06 6.31 ~0. 16 0.44 0.06 90.02 
EXPE 0.33 0. 79. -0..30 =). OF -~0.23 -0.07 
EXPL -0.36 4.01 06.78 0.2% 0.36 -0.04 
FACT 0.24 0.24 0.31 0.78 0.43 -0.27 
HYPO “0.34 -0.80 0.27 -0.20 -0.35 0..29 
IMAG -9.23 -9.0% 0.28 0.06 -0.16 0.15 
INTE ~0.29 -0.25 9.15 =—9.95 -0.78 0.27 
INVE 0.33 -0.29 -0.26 -0.94 —~0,06 “0.73 
LAW G. 1% +0. 97 —Q.48 0.06 0.24 -0.25 
PROO 0. 3t O.37 -0. 176 0.38 0.35 -0.26 
PUZZ 0.05 $2.30 =0. 56 “ene “Gene Wo3t 
OUES -$.51 -0.24 -G..01 -0.44 0.05 0.25 
THEO ~Q.25 “0.7%. 0.7% —~0.28 -0.36 9.27 
CONC 0.04 9.15 -0.18 0.14 09.14 -0.05 
DISC 0420 --0403.--0~. 32 0.74 0.05 90.03 
EVID 9.26 «0.4% —¢.27 0.34 0.28 0.42 
EXPE O.27 «8525 -0.05 ~0.14 0.14 0.31 
EXPL 0.07 0.04 -0.14 0.98 6.37 -0.81 
FACT 8.71% 0.02 -0.24 0.26 0.27 0.34 
HYPO —9.39 -0.900 0.27 -$.08 -0.49 -0.20 
IMAG -0.32 -0.30 0.0 -0,29 -0.239 —-0.25 
INTE ~0533 . 0.73: -G.,70 =S.0F 0.72 -0508 
INVE 0.29 0.24 0.40 =—O.0S 0.05 “Gat0 
LAW 0.32 0.09 —0. T2 0.03 -0.00 0.12 
PROO 0.24 0.08 -0.25 0.46 0.25 0.28 
PUZZ =J.23 ~9.73. 09.32 =G. 94 -—0.489 -G.19 
QUES -0.238 -0.37 0.51 =f,28 —O.67 =—0.527 
THEO -0.25 @.79 0.28 0.07 -0.34 -0. 30 
CONC 6.22 -G. BH 6.36 ~0.58% 0. 70 -—@.39 
DISC 0.05- 8627 0. 20 Oe22 OS 20> eB e20 
EVID G.37 0.30 0.171 G.94 -G..78 -0.32 
EXPE 0.04 9.35 0.07 6.37 0.79 °0.43 
EXPL -9.06 6.26 0. 71 “S27 (8.09 -O.07 
FACT 0.33 O..97 -@..11 <0. ©0223 —0.07 
HYPO -0.48 -0.35 -0.40 =O, 70 =-0.49 -0. 23 
IMAG -0. 09 -O.70 -0. 21 -$.02 —-G.323 0.26 
INTE +0. 323 -G.55 -0.06 =0.07 =0.02 -0.09 
INVE -0.06 0.10 -0. 13 0.32 0.368 0.02 
LAW 0.76 -@.22 -0.08 %. 14 -F.09 -0..08 
PROO O37 0.06 -0.27 -0.16 09.26 0.16 
PUZZ -0.19 0.24 0.42 0.45 0.04 06.12 
QUES -~9..903 -G.30 -0.34 Tt =—0.493 -0.36 
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TABLE L 
CARROLL SOLUTIONS OF TUCKER SUBGROUPS 
CAUS GROUPS 1,2,3,4; CAU12 GROUPS 1,2 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC “Ga8t Qe20 "0.4495 0.92 @.20 0242 
DISC -~0.13 0.18 -0.19 =0.03 G.26 =0543 
EVID 0.53 -0.27 0.969 0.20 ~0. 060 -—0.28 
EXPE “0.07 -G,0% —0. 62 S19 Ga29 =Gak2 
EXPL “9.26 0.30 0. 26 9.02 0.71 0.25 
FACT 0.43 O19 0.11 Gs27T -0.48 -0. 799 
HYPO -9. 09 =0,49 ~ 0.497 ~O597 0.24 O.28 
IMAG -0.09 -0.20 05.97 “9.08 -U. 19 -0.07 
INTE “0,12 -~G.a0 9.05 -9.52 0.07 -0.08 
INVE =“0.08 0.07 -0. 60 0.715 9.28 -0.31 
LAW 0.174 -0.11 0.06 “500 “0,46 0.79 
PROO 0.48 0.26 0.07 Ge29 -0.36 =-0.22 
PUZZ ~0. 21 ~G.09 “0.18 0.42 ~0.07 0.46 
QUES —O2260 702 16° °OS02 0.78 0.23 0.36 
THEO “9,908 =9.55 0.95 =G5492 0.07 -0. 413 
CONC =§.32 -0. 78 —0. 10 =§,02 -0.35 -—0.40 
DISC 0,71 -0:.06 -0. 83 0240 0.76 -0.58 
EVID =“$.25 0538 “0.2172 0.30 -0.29 6.03 
EXPE GO. 17 -0.06 =0.52 Sot2. Usaz OO. 97 
EXPL -~G.3290 -0.07 0.07 ~-Ys2% —-0.0F “O.17 
FACT -0.27 0.398 0. 77 0.19 -0.45 0.20 
HYPO 0490 =—O539 © f.s22 -0.29 O.27 ~0.170 
IMAG Ges ~9.26 9.21 “Gat “UstS. =9208 
INTE -Ue2d =0.30 Ge 71 ~G. 30 °0. 08 -0.33 
INVE 0.26 0.02 -0.50 6.25 0.20 0.394 
LAW 0.07 0.45 0.17 0.26 =0.27 0277 
PROO =~9.22 “8.38 0,45 O2.30 =-Gss7? -GatG 
PUZZ 0.46 -0.10 0.16 —Go25 G.49° “Gera 
OUES 0.48 -0.78 0.11 “G20 8.18 -Os28 
THEO -0.08 9.0 0.11 =-0. 18 0530 ~0. 08 
CONC Ge 44 “0.39 =0.25 =Go25 —-Us2G =0.27 
DL SC 0.05 -0.70 =0.455 O.07 ~G.26 =-0.25 
EVID 0425 -O04.15 0.423 Ga35 ~U. 37 -0.277 
EXPE 0.24 0.32 =-0. 77 -0.09 -0.09 -0.42 
EXPL -§0.10 -0.25 -0. 37 —O0.77 =-0.47 0.26 
FACT 0.20 -0.28 0.43 6.25 -O. 78 O27 
HYPO -0.09 0.30 -0.06 “G29 “U2 “e726 
IMAG ~0.48 0577 0.15 -9.26 0.08 ~ 0.20 
INTE -0.46 -0.13 -0.11 “0.25 ~0.49 0.17 
INVE 0.23 “0.88 -0.42 9.86 0.49 -9.35 
LAW 0.370 =0.27 0.38 0.40 -0.05 ~0.28 
PROO 0.26 -0.23 0.34 0.29 =8599 —0.i18 
PUZZ 0.10 0.32 90.04 -0.19 0.54 0.40 
OURS Oett- sss. ~ ts T' =-O629°° Gs 16° 0525 
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TABLE L (CONT.) 
CARROLL SOLUTIONS OF TUCKER SUBGROUPS 
CAU12 GROUPS 3,4; CASS GROUPS 1,2,3,4 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC ~O.15 =—O.37 -0.24 =? 0S -0723 —O. 08 
DISC -0.0) -0.24 0.346 =O.Tf ~0,. 73 -0.44 
EVID “0.35 0.79 -9..12 0.45 0.26 0.172 
EX PE 0.01 -0.24 0.00 =—9.39 ~0. 45 —-0.52 
EXPL 0.03 -0.10 -0. 42 0.08 -0.93 -0.1 
FACT =0.425 ©.38 0.55 0.49 0.13 0.08 
HYPO 0.78 ~0.28 -0.07 “9.74 ~0.. 3% —-0. 438 
IMAG 0.43 0.54 -0. 16 =O, 30 -G.901 0.28 
INTE 0.27 -0.95 -0.31 =“3.03 -0.50 9.529 
INVE -0.905 0.72 0.39 —U.22 “0.3% —0..05 
LAW ~0.3223 0.289 0.06 ~9.07 9.92 G.22 
PROO -0.88 0.97 -9.17 0.5% 0.26 0.190 
PUZZ 0.24 =-0.08 0.85 “(28 8,18 0. 39 
QUES 0.79 =-0.79 0.34 -O.23 O05 74 «0.26 
THEO 0.27 -G.24 -9. 11 =—..06 “G22 0 ~Us3d79 
CONC -0.07 -9.76 6. 05 =-Gs19 Go03 =0.29 
DL SC =—9. 23 —-0.38 ~0. 05 -§.34 -0.36 0.02 
EVID =G.91 .37 @.42 —“Ooet T.2e VlSG 
EXPE -0.08 -0.50 0.13 9.32 -0. TH -0'..33 
EXPL -§.70 0.06 -0.09 9590 O532 -0. 13 
FACT -0.30 0.28 dG. 77 =O0..F1 O225 0.26 
HYPO 0.30 O.77 -9. 480 G237 2.98 =O. 497 
IMAG 0. 32 «0.05 -0..23 0.36 -0.27 0.74 
INTE 9.96 0.13 -0.27 0.15 0.20 -0..16 
INVE ~0.03 -0.489 0. 10 -0.27 -0.24 -0.40 
LAW =G.20 G.23 ©0203 O.09 -0.40 0.32 
PROO —9.22 0.27 0.38 —G.032 9239 .-0..93 
PUZZ 0.54% -0.02 9.78 6.29 -0.36. 0.09 
QUES 5.51 -0.04 0..15 Sate Bath =o 5 to 
THEO -0.07 G.13 -0. 54 0.33 0.28 -0.92 
CONC =O 78 “S272  G.29 -§.27 ~-G.346 —-G.39 
DISC 0. 1 -9..30 —0. 123 -G.08 -G. 07 —O.21 
EVID 0.82 -0.34 -0.18 “220 Sosa 0.09 
EXPE 0.09 @.42 -G.259 Qo22 -O.02 -0.248 
EXPL -0.18 0.08 0.41 —“G.20 -0. 37 -0. 41 
FACT 0.40 -G.38 -0..09 -G.27 G.44 -0..08 
HYPO -0.24 0.017 0.22 0.06 -0.25 0.42 
IMAG —-§..33 -G.. 1 —&.52 0.30 -0.04 0.45 
INTE -0.19 0.09 0.15 —Oot9 -Gagh —e 72 
INVE o.. 72 G39 -o. 17 0.29 ~G.05 -0..22 
LAW G. 12 ~-0.28 -O..02 —9. TF G36 “0.26 
PROO G39 -G.40 0..03 =(.28 9.52 ~0..09 
PUZZ -0.36 -0.14 -—0.37 0.44 0.04 -0.18 
QUES -0.21 0.16 0.18 0.40 -0.03 -0.22 


THEO -0.16 0.09 0.28 —P.0T =7.20 ~0..50 
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TABLE L (CONT.) 
CARROLL SOLUTIONS OF TUCKER SUBGROUPS 
CAS12 GROUPS 1,2,3,4; SJ9 GROUPS 1,2 


DIMENSION 
CONCEPT 1 2 3 1 ys 3 
CONC C205 -G.37 0.2.39 S220 “G2 Hoe 
BiSE G.23 -0,05 0.20 G.92 ~G.29 0.96 
EVID “9.83 Goss -O..18 -0.43 -0.04 0.38 
EXPE G 3% —-O. 1S -O. 24 OTF =C.. 98 ~O.08 
EXPL =0. 03 -G0.27 814 0.03 0.38 -0.43 
FACT -G.47 0.22 6.14 “S38 et 8.08 
HYPO 0.26 G09 -0.179 t.2e —O.35 —0. 39 
IMAG G.05 0.70 -0.. 72 0626 GO. 33 -G..28 
INTE —0. 72 -G23o —6s.355 0.18 9.37 @. 179 
INVE G.37 0.78 Q.79 ~“O.12 -Go 28 0.03 
LAW 0.04 0.41 -0.08 =§.39 @G.94 6.01 
PROO 0.48 .32 G%..20 “2a “9.09 «O.36 
PUZ -G... 08 —-6.36 —0..19 29 9.19 0.14 
QUES O. 18 ~Gi 2 GC. OF 0.92 0.76 -0.28 
THEO On27 W208 -.01 —1.07 0.279. -Go85 
CONC ~0.14 -—0.28 -0. 06 “Gene Vata -On29 
DISC S202 =—0. 74 -8. 15 -0.04 0.00 -0.45 
EVID ~0.36 0.31 @.76 =O.42 "-G.00 “2.27 
EXPE 0.40 0.69 0.38 9.06 -0.358 -G6. 371 
EXPL —~0.02 0.060 -0.34 =-§.901T 0.417 =-0.34 
FACT “0.33 .38 -%.032 -0.36 0.05 06.38 
HYPO G.49 0.78  0..07 O.2% -6. 72 -@.07 
IMAG =O. 02 ~G.239 -9. 35 6.32 98.20 0.75 
INTE 0.08 -6. 16 =-0.33 0.19 0.48 -0.02 
INVE S240 -0..13 0.496 0.95 -0.47 -0.06 
LAW -05 08 G2 09 -—G. 04 =-9.28 0.08 0.27 
PROO “0.37 .0.39 0.05 =O.42 =O.00 -O.22 
PUZZ =—0.059 -0.49 9.179 G131 -0.25 0.71 
QUES =—9.07 ~Oc2e - 6.5.23 9.22 —-9.24 =O. 15 
THEO 0.38 0.28 -0.39 G22 8.39% 0.32 
CONC -0.25 ~0.25 0.42 G.30 ~0.38% -0.29 
DISC -0.00 -0.06 0.40 G223 -0. 08 © 0.07 
EVID =0. 75 -0.27 -0. 06 9.26 0.26 -0.09 
EXPE 0.93 9.29 -0. 29 O.76 -9.37 0.34 
EXPL —§..24 -0.22 6.15 =U.06 =-0.03 -0.37 
FACT -9.29 -0.20 -0.33 G.23 0.36 -0.18 
HYPO 0.41 0.08 0.13 -G.9f =-G..45 ~0.:905 
IMAG 0.50 -0.78 0.03 -~0.88 -§.07 -0..13 
INTE -0.13 -0.10 0.18 ~9. 98. -0.37% -0.38 
INVE -0.04 0.30 0.02 S82 ~0.39 =0..03 
LAW -0.30 -0.06 -0.25 Ont2 0.43 -0. 75 
PRCO -0.25 -0.16 -0.40 6.28 0.36 -0.01 
PUZZ 6.20 0.868 -0.32 -~9.35 -0.99 0.52 
QUES 0.22 0.44 -0.12 “S.36 0.72 0.58 
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TABLE L (CONT.) 
CARROLL SOLUTIONS OF TUCKER SUBGROUPS 
SJ9 GROUPS 3,4; SJ12 GROUPS 1,2,3,4 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC ~0. 78 -6.23 0.901 -0.30 0.18 0.10 
DISC -0.16 0.75 -—0. 10 —-0.36 ~9.15 -9. 09 
EVID —O0.3%9 0.35 -0. 905 -0. 77 0.30 -0..73 
EXPE 0.15 -0.00 -0.36 -§.07 -0.33 -0.26 
EXPL -0.26 0.14 0.05 —§.22 ~0.90 -0.99 
FACT -9,.20 -§.05 0.07 “0.27 ~G.30 =U. 27 
HYPO V.84 =9.. 99 -&..22 6.23 -0.22 0.27 
IMAG 0.22 -0.54 0.43 0.35 ~0. 17 ~9.45 
INTE 6.38 -90.37 ~0.70 -0.07 G.08 0.55 
INVE 0.17% 0.92 -0..45 0.09 -0.27 -0.238 
LAW —-0.26 0272 0.28 0.08 0.46 -0.22 
PROO -0.39 0.25 -0.62 O86 “G2? —O21S 
PUZZ 0.23 -3.56 =-0.21 0230 ~G.3) -0. 47 
QUES 0.36 —9.23 —0. 34 O.33 -—O.28 -9.-42 
THEO G.22 -0.08 0.2% 9.428 0.78 G.20 
CONC 9.29 -—G.03 -0. 791 —G273 G9. 24% =O. 26 
DISC —9. 06 —0..06 -—0..22 -J.%0 ~0.823 <—8..05 
EVID 0.24% 0.12 0.05 =—Goa8 —Us0? ~G.02 
EXPE —7.15 6.32 0.35 8.20 0.713 -0.09 
EXPL 0.76 60.20 -—0. 31 =O5.49 ~U.340 -0..29 
FACT G32 0.72 -0. 13 =So39 “Usa —Gl 13 
HY PO —-O.39 G52 -G. 10 0.28 9.15 6.74 
IMAG —~O%o =O, 77 ~~. 38 0245 0.07 0.32 
IETE U601 -0. TS -0.29 Gate Ceet -te33 
INVE ~G201 -0.15 9.50 =0.76 C238 -O.23 
LAW 0.36 -0.41 0.174 =0..20 -0.97 —-0..06 
PROO 0.28 0.06 0.07 =0.25 ~G.29 -0. 16 
PUZZ —.37 -G.39 C..08 0.23 0.06 0.55 
QUES -$. 28 -O.27 G.32 0.10 -8.05 0.357 
THEO -0.74 0.28 -0..06 0.40 -0.26 -0.28 
CONC -0.2% 0.20 0.68 =-G520 Os27 -0. 16 
DISC 0. 0° -G..73 8.26 =-o,09 -G. 0 —-o. TF 
EVID -0..W G.Tt 0.31 -—9,.20 O29 =0.18 
EXPE 0.78 -0.04 90.02 0.719 0.01 0.06 
EXPL —-O. 77 0.08 -O.08 =-0.08 G.3T “0. 07 
FACT -9..36 C210 Daz =-0.46 0.76 -0.13 
HYPO G22 278 -%.24 C..08 -G.59 —3. 15 
IMAG G.. 12 -d..53 -—6.50 0.32 ~-G. 27% 8.74 
INTE ~0.26 -0.03 -0.40 0.32 8.30 -0528 
INVE G31 2.78 @..25 O.27T G.77 -0.06 
LAW -0.25 -0.907 0.284 -0.43 G13 9.20 
PROO -0.27 8.1 0.27 =Go2t Gas2 =O. Fo 
PUZZ 0.44 -0.66 -0.08 G27 -G.29 0.45 
QUES 0.38 0.08 -0..13 0.76 -0.25 99.65 


THEO 93 0.34 -0..29 Cn2k “Ge 29 -O.33 
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TABLE L (CONT.) 
CARROLL SOLUTIONS OF TUCKER SUBGROUPS 
SD9 GROUPS 1,2,3,4; SD12 GROUPS 1,2 


DIMENSION 
CONCEPT 1 2 3 1 2 3 
CONC =O. 0h =G.85 9.77 0.90 -0.18 0.00 
DESC -0.34 0.30 -0.10 GO. 0% ~0.50 -—0. 25 
EVID “O,.29 (2.22 +0.99 0.56 -0..07 -0.06 
EXPE -0.05 0.37 0.46 o.0% 0.0% .0.38 
EXPL -0. 05 -6..18 =0. 91 0.06 -0.9% —0.23 
FACT =9..26 -0. 93 -0..69 0.29 -0.33 -0. 26 
HYPO 0.44 0.19 -0.27 =Hs 10 0.42 GO. 95 
IMAG 0.10 0.02 -0. 40 =O. 02 -O..07 - 8. 
INTE h.13 ~0.27 —i.22 —O. 1 G28 257 
INVE 0.10 0.20 0.44 “0.09 O.08  —O.04 
LAW -0.33 0.79 8.18 o.05 -2.30 —0..99 
PROO —“Qo2f On 04 Ge 32 0.47 -0.30 =-0.02 
PU ZZ G22 0,28 -%. 71 =Uo28 Bes’ =O. a5 
QUES 6.39 -0.26 0. 1% =,4% $22 —-0,..85 
THEO §.24 —-0.. 77 —0.37 “0.26 0.29 0.79 
CONC Goit =—0.79 -0.25 “$178 -O.23 -0. 49 
DISC “9.37 -0.26 0.2.34 =0..90 =G.10 D229 
EVID =0.20 -O225 0.05 =—H.24 =“U,-42 —05491 
EXPE =G.32 05.06 -—0. 906 “Wade  O. 06 =. 37 
EXPL GS te =0057) =05.42 =92.908 -O.27 0.41 
FACT =G.17 —0. 19 =O. 20 =§. 10 —Q.37 O.71 
HYPO 0.39 0.22 =-0.34 G20 O82. -0..49 
IMAG Go3!) —U.42 -0.08 0.19 0.171 0.40 
INTE G.32 9.07 -0.09 =O. 90) Uma d 0.9 
INVE -Gs441. 0551 0.42 “215 Datla —Uo2o 
LAW 0.06 -0.03 0.86 -0. 76 -0.20 0.1% 
PROO -Qea9 0.39 =U. 0% =O. 11—U.08.—-G. 6 
PUZZ -Q.22 Us47° D202 O77 0.18 -0.904 
QUES Jeol. Usses Gact 0.36 0.07 0.38 
THEO 0.43 0.2.29 —0.38 “40  Uetl “Varese 
CONC -0.45 -0.93 0.43 -§.24 -0.03 0.22 
DESC O.27 =0525 0.07 9.09 6.22 -0.36 
EVID G.35 ~0.37% -0.25 =9,30 0.29 -0..31 
EXPE G..39. “CU. 16 -0.:59 200 G228 -G.-1F 
EX PI -0.20 -0.18 90.2% “0.54 “8.28 —0. 36 
FACT -0.05 -0.14 0.05 -O0.32 .Q432 0.25 
HYPO -0.20 0.54 0.10 G49. -9.909 - 0.26 
IMAG 0.08 0.06 -0. 34 0637 -9.33° 0.09 
INTE =—9,05 6.33 -0. 29 =“Qo25 =0.37 0.80 
INVE 0.28 -6..05. 0.25 0.82 0.29 ~9.27 
LAW 0.03 -0.16 90.903 0.0% 0.139 =-0.03 
PROO -0.08 -0.25 0.07 ~G.2F 8.26 -0.29 
PUZZ 0.3% 0.45 0.20 0.36 <-0.02 0.172 
QUES =9.92 .29 -8279 0.27 -0.20 9.19 
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TABLE L (CONT.) 
CARROLL SOLUTIONS OF TUCKER SUBGROUPS 
S$D12 GROUPS 3,4. 


DIMENSTON 
CONCEPT 1 2 3 1 2 3 
CONC -§. 26 -0.77 0.02 -0.05 0.40 0.36 
bE SC =O. 32) Geto) =0043 =0,.03 G.22 —-0.31 
EVID “Oe 30 Oc23 “0209 0.65 0.16 0.09 
EXPE —9507 0536 9234 OetZ Ga. t2 =9.25 
EXPL 0.02 0.24 -—9. 02 =—Oo 1 Oct 8.02 
FACT -O533 0.28 ~05 33 Vaan ~0.29 ~0. 18 
HYPO 0.448 -0.34 -0.18 ~0.27 -O0.84 0.47 
IMAG 0.77 -0.16 —-0.06 =—06 74 00497. 0.33 
INTE 0.32 -O.7T7 0.00 20579 =“O..13 “0.08 
INVE G..41 . G.28 GatzZ 0500 =—0. 12 =0535 
LAW =G. 20 0.05 —0220 “02 CG. 15 —G.0% 
PROO =O.32 507 0.05 0.47 0.20 -0.18 
PUZZ 0.03 —G283 -0.499 “O76 0530 -—-0.,33 
QURS 0.096 -O277% 9.274 =0. 19 —V.ce8 02:47 
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